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social -emotional -behavioral competence. The model has proven effective in 
inclusive settings with children who have social, emotional, or behavioral 
disabilities, and those who have additional developmental delay, autism, or 
other disabilities. Components of the model have been integrated into early 
childhood general pre-academic curriculums and in natural settings. Family 
involvement such as parental participation in assessment, program planning, 
and simultaneous home implementation of model practices is an integral aspect 
of the model. The project assists early childhood and local child care 
programs in replicating components of the model in natural, inclusive, or 
pull-out settings for children (birth to age 8) with 

social-emotional-behavioral disabilities, and their families. Recognizing 
that effective implementation depends upon the knowledge and skills of the 
adults who work directly with these young children, the project assists 
participants in acquiring specific skills to foster 

social -emotional -behavioral growth. Emphasis is on model applications in 
typical daily activities such as social play, social language, listening and 
responding, creating, imagining, playing, and participating. Specific 
outreach activities include: (1) dissemination of information and general 

training about the model to early childhood personnel and families of 
participating children; (2) planning and model implementation at selected 
replication sites, with new staff development materials suitable for on-site 
and distance learning via Internet and satellite; and (3) evaluation of 
project impact on the proficiency of participating personnel and families, 
their evaluation of helpfulness and effectiveness of the outreach project, 
and the progress of participating children. (SG) 
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EXECUTIVE SUMMARY 



PRMward No: HO24D70035-98 



DEVELOPMENTAL THERAPY-DEVELOPMENTAL TEACHING: 

An Outreach Project For 

Young Children With Social-Emotional-Behavioral Disabilities 
(CFDA No. 84.24 D) 

October 1, 1997 — September 30, 2000 

Final Report 

This project, Developmental Therapy-Developmental Teaching: An Outreach Project for Young 
Children with Social-Emotional-Behavioral Disabilities (CFDA No. 84.024D), provided outreach 
assistance to programs serving children, ages birth to eight, with severe social, emotional, or 
behavioral disabilities (and also those with other disabilities including autism, when behavior was 
also a problem). Programs receiving assistance from this project were providing services to 
youngsters in inclusive general education, in inclusive special education, in special education 
classes, in psychoeducational programs, and in community and other natural settings. The original 
goals remained unchanged during the three years of the project. 



Outreach Outcomes: Goals Accomplished 

1 . Personnel with increased understanding of social-emotional development. 

2. Personnel with increased skills to foster social-emotional development. 

3. Children with increased social-emotional-behavioral competence at quality 
replication sites. 

4. Effective outreach project activities and products. 



Project Activities 

Project activities focused explicitly around the outreach mission: To assist parents and practitioners 
in early childhood and child care programs in effectively implementing proven practices of the 
Developmental Therapy-Teaching curriculum model. Outreach services included dissemination of 
information about the model (Management Objective 1); consultation and planning for model 



0 



Executive Summary, i 



workshops, in-class tutorials and in-depth follow-up (Management Objective 3); coordination with 
state and national agencies (Management Objective 4); outreach assistance for professional 
development through workshops, distance learning, and teleconferencing (Management Objective 
5); preparation of inservice instructional sequences and media for use with local programs 
(Management Objective 6); design of new outreach activities, including a training-trainers program, 
and modification of existing outreach strategies to meet changing needs of personnel in multiple 
settings (Management Objective 7; Management Objective 8 in Year 2 report); and evaluation of 
project accomplishments in meeting the needs of programs and individuals at each site, with 
particular focus on improving the performance and effectiveness of the service providers 
(Management Objective 8; Management Objective 7 in Year 2 report). 

On a year-by-year basis, the project worked with 5 programs during the first year of the 
grant; 16 programs the second year (5 continued plus 1 1 new programs), and 18 programs in the 
third project year ( 1 3 continued plus 5 new programs). Details of each management activity and its 
accomplishments are provided in the following sections. 

Project Outcomes 

At the end of the three-year period, the project exceeded anticipated outcomes for each management 
objective. Through dissemination activities, the project reached approximately 3,800 individuals in 
38 states, the Virgin Islands, and 15 foreign countries, seeking information about the model and/or 
outreach assistance. More than 1 ,992 individuals received inservice training through professional 
development workshops and/or intensive in-depth training for model implementation. Local needs 
assessments for planning model implementation was provided to 20 programs in 7 states. In these 
20 programs, 346 individuals serving 585 children with special needs received in-depth, extended 
outreach assistance during the three-year period. The project was on location at replication sites for 
224 days. Some days were conducted by one associate and some were as many as four. 

Over the project period, outreach activities were coordinated with 7 state agencies. New 
materials/products included (a) new video productions for introduction to the model, (b) computer- 
aided materials, (c) new training materials for trainers-in-training, and (d) new training materials for 
skill practice by site personnel. Additionally, at the end of the final project year, 6 leadership 
participants had completed the trainers-in-training certification requirements. Figure 1 provides an 
overview of these project accomplishments. 
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Project Effectiveness 

Project effectiveness was defined as (a) participants demonstrating significantly increased skill in 
using the specified practices to foster the social-emotional-behavioral development of children in 
their local settings; (b) young children with disabilities making significant gains in social-emotional- 
behavioral development in programs with demonstrated quality replications of the Developmental 
Therapy-Developmental Teaching mode, and (c) outreach activities, services, and products judged 
by recipients to be effective in meeting their needs. 

At the completion of the three project years, 20 program sites received extensive site 
development, technical assistance, and replication services. These services impacted directly on 346 
direct service providers and parents, and 585 young children with disabilities (outcome projections 
were for twelve to eighteen program sites to have received extensive site development, technical 
assistance, and replication services with direct impact on approximately 200 direct service providers 
and parents). Observational ratings of actual performance of a representative sample of direct service 
providers indicated that 84% acquired a proficiency score of Adequate or better by demonstrating 
basic practices necessary for model implementation. Of these, 87% achieved higher proficiency 
scores at Effective or Highly Effective levels of proficiency. In addition to the outcomes specified 
in the original project proposal, we began in Year Two to emphasize training of leadership 
individuals to become on-site trainers — our developing Regional Associates Program for training 
local trainers (see Management Objective 7). It was anticipated that half of the participating 
programs would train a local leadership individual to provide continuing model dissemination, 
outreach assistance staff development after the grant funding period. This projection was exceeded; 
13 sites had leadership personnel participate in the Training of Trainers Program with twenty 
individuals accepted for the Training of Trainers Program. Six of the twenty have become 
Developmental Therapy - Teaching Regional Associate Instructors. 

Measures of satisfaction of participants with their training experiences indicate that project 
activities met their needs, and most respondents indicated considerable gains in understanding and 
skills. Almost all participants also indicated a need for further training or more time with project 
instructors on-site. Workshop effectiveness, assessed by 1 ,646 participants (including participating 
parents) received average ratings of 4.37 to 4.64 on a scale of 5 {Highly Satisfied) to 1 {Not 
Satisfied), indicating high degrees of satisfaction. Satisfaction of direct service teams, assessed 
through post-project anonymous questionnaires, indicates levels of satisfaction from above average 
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through post-project anonymous questionnaires, indicates levels of satisfaction from above average 
(ratings >3.0) to highly satisfied (ratings of 5.0) on all four project training dimensions: workshops, 
observations in their classrooms, debriefings for feedback, and written feedback. Views of 
leadership trainees about their satisfaction and usefulness of their project experiences was assessed 
through a focus group discussion. The participants held high opinions of their experiences both 
professionally and personally. 

Overall effectiveness of the project was obtained by interviewing local coordinators to assess 
the extent to which participating programs acquired the basic elements for model replication. Of the 
20 sites that participated in evaluation of child progress, all were rated at the Basic Implementation 
level or better, and three sites achieved the highest Exemplary Model Demonstration level. 
Leadership individuals in the local programs who have successfully completed the RA training of 
trainers program can continue to provide staff support, train new personnel, and document program 
effectiveness. 

Together, these evaluation results indicate that the overall project mission to improve service 
for children and youth with severe social-emotional-behavioral disabilities was achieved with 
distinct and measurable performance indicators. Project goals were effectively accomplished and 
exceeded anticipated outcomes in the original proposal. 



***** 




Constance A. Quirkyrh.D. 

Project Director 
University of Georgia 
College of Family and Consumer Sciences 
Developmental Therapy-Teaching Programs 
Athens, Georgia 30601 



December 20, 2000 
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FINAL REPORT 

DEVELOPMENTAL THERAPY - DEVELOPMENTAL TEACHING: 

An Outreach Project For 

Young Children With Social-Emotional-Behavioral Disabilities 

CFDA No. 84.24 D 

October 1, 1997 — September 30, 2000 
THE INTERVENTION MODEL 

The Developmental Therapy-Teaching curriculum provides a framework for guiding social- 
emotional. development and responsible behavior in children and teens. It matches a child’s current 
social, emotional, and behavioral status with specific goals, objectives, behavior management 
strategies, curriculum materials, activities, and evaluation procedures. It also defines specific roles 
for adults to facilitate a child’s development. The curriculum sequentially spans social, emotional, 
and behavioral development for children and youth from birth to 16 years. 

The curriculum has four areas; Behavior, Communication, Socialization, and (Pre) 
Academics/Cognition, to address four essential human activities — doing, saying, caring, and 
thinking Within each of these four areas, specific teaching objectives follow developmental 
sequences for social-emotional competence and responsible behavior. Specific curriculum activities, 
management strategies, and adult roles define the ways the model is implemented for preschoolers, 
school-aged children, and teens. 

Three measurement instruments provide the core evaluation measures for this curriculum. 
The Developmental Teaching Objectives Rating Form-Revised (DTORF-R) is a 17 1-item assessment 
instrument used to obtain a profile of a child’s social-emotional-behavioral status. It identifies 
specific objectives for social-emotional competence in an Individualized Education Program (IEP), 
Individualized Family Service Plan (IFSP), or Individual Transition Plan (ITP). The rating process 
is used also for a functional behavioral assessment, provides a profile of current strengths as well 
as areas of difficulty, and is used at repeated intervals to evaluate child progress. 

The Developmental Therapy Rating Inventory of Teacher Skills (DTRITS) has four forms 
specifying the basic adaptations in practices for model implementation in four large age groups: 
infant/toddlers, preschool, elementary school-aged, and in middle/high school. The DTRITS 
provides an observational rating of an adult’s current performance skills, serves as a needs 
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assessment for planning inservice training, is the basis for tutorial feedback, can be used as a self- 
guide for model implementation, and documents acquisition and maintenance of skills over time. 
DTRITS data also provide measures of replication fidelity at sites attempting model implementation. 
An Administrative Support Checklist contains 41 basic administrative elements associated with 
levels of program quality in model replication. Previous studies of model effectiveness have shown 
that certain minimal levels of administrative support were necessary to support successful 
performance by direct service teams in classroom settings as measured by the DTRITS during a 
school year. 

The evaluation plan uses these three instruments to obtain measures of both qualitative and 
quantitative assessment of outreach activities and the optimal settings/conditions for achieving the 
greatest results. These measures of trainees, children, and programs were analyzed for evaluation 
of outcome effectiveness. The benefits from such analyses are these: 

• Formative feedback to individual participants re-focuses training so that learning experiences 
can be redefined, reinforced, revised, and replicated. 

• Summative feedback documents project accomplishments and permits staff to examine the 
quality of outcomes. 

HOW THE PROJECT GOALS WERE ACCOMPLISHED 

The Developmental Therapy-Teaching Programs is an outreach unit of the College of Family and 
Consumer Sciences at the University of Georgia, Athens, Georgia. The unit enjoys outstanding 
administrative support and working relationships with the Office of the Vice President for Services 
and Outreach, Dr. Eugene Younts; and in the College, with Dean Sharon Nickols, Associate Dean 
Christine Todd (Years 1 and 2), and Associate Dean Tom Rodgers (Year 3). The unit is comfortably 
housed off-campus due to a critical space shortage at the University, but is able to connect directly 
to all of the on-campus support systems. Appendix A illustrates the administrative organization of 
the unit within the University. 

During the three years of this grant project, the unit received additional grant support for 
other outreach, training, and service activities from the Georgia Department of Education, U. S. 
Department of Education, Office of Special Education, Early Education Programs (CFDA 84.324R 
and CFDA 84.325N); State of Washington, Department of Social and Health Services, Division 
of Children and Family Services; and local public education and community service programs. 
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Project staffing went through several changes during the three grant years. The original 
Project Director, Karen R. Davis, became ill during the first year and subsequently went on 
disability status and died. Dr. Mary M. Wood, retired Professor Emeritus of Special Education and 
founder of the Developmental Therapy-Developmental Teaching model served as Interim Project 
Director. Dr. Connie Quirk, an experienced and certified National Instructor for Developmental 
Therapy, joined the project in Year 1 as Senior Trainer and subsequently became Project Director 
for Year 3. Dr. Faye Swindle served as part-time Senior Training Associate for the entire three years 
of the project. A second Training Associate, Julie Hendrick became ill also during the first few 
months of the project and was retired on disability. Her part-time position was filled by Diane 
Wahlers as Coordinator of Outreach and Distance Learning during the second and third grant years. 
Coordinator for Dissemination Betty DeLorme and Office Manager Debbie Huth served the project 
part-time throughout the three grant years. Elizabeth Carbone served as Evaluation Coordinator 
during Project Year 3 . In addition to this core staff, the project was able to obtain the services of four 
highly experienced and certified National Instructors in Developmental Therapy as adjunct 
staff/consultants to assist with in-depth training at selected field sites. These were Dr. Susan Galis, 
Dr. Mary Leiter, Dr. Bonnie McCarty, and Rosalie McKenzie, a specialist in services to young 
children with Autism Spectrum Disorders. In addition, Dr. William Swan and Dr. Douglas Flor 
provided services for project evaluation. 

The success of the project as well as its mission depended on bonding project goals and the 
agencies' requirements and commitments. Programs and agencies seeking to improve their services 
needed all available information about resources and options. With awareness of this, the project had 
extensive communications with each potential site during planning phases so that expectations of 
administrators and direct service providers were matched to the outreach assistance as nearly as 
possible. This overarching principle guided project activities while keeping efforts focused on the 
particular project objectives and outcomes as specified in the original proposal. These activities and 
accomplishments are reported below, according to project management objectives. 

Management Objective 1, Dissemination: To disseminate information about the outreach project 
and the model. 

The goal for this objective was to provide information about the model to individuals, 
parents, and early childhood practitioners concerned with meeting the special needs of young 
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children who are troubled, those with emotional or behavioral problems, and those at-risk. During 
the first two years, the project worked collaboratively with another OSEP project which provided 
similar outreach and technical assistance to programs serving older, school age children to age 16 
with severe social-emotional-behavioral disabilities. In its final year, the project worked 
collaboratively with a third OSEP project which provided leadership training to onsite supervisors 
of programs serving children and youth with severe social-emotional-behavioral disabilities. These 
projects shared costs for information dissemination about the core model components and how to 
access outreach services. These cross-project efforts provided greater combined output for 
addressing the needs of children and youth with social-emotional-behavioral problems from birth 
to age 16. 

Project information was disseminated to over 3,800 individuals in 38 states. Internationally, 
the project received requests for materials and information from Australia, Canada, China, Ireland, 
Italy, Israel, Germany, Japan, Lithuania, New Zealand, Norway, Russia, Scotland, Singapore, 
Taiwan, Ukraine, and the Virgin Islands. Telephone, fax and e-mail communications were additional 
forms of dissemination used extensively for exchange of information and consultation. 

Of the disseminated print materials, "awareness" packets of information (including a 
newsletter and new brochure) were distributed to 175 individuals in 20 states and over 1700 were 
distributed to participants at professional conferences and workshops. The new 5-section foldout 
brochure highlights the model elements as it is applied across the age spans (Copy attached at the 
end of this report.) Over 3,000 bookmarks were distributed at conferences and training sessions to 
introduce our website (Copy attached at the end of this report.) Five issues of the newsletter were 
mailed to 2,200 individuals in 38 states. The newsletters included front page feature articles 
concerned with the psychoeducational principles that are utilized in the Developmental Therapy- 
Teaching approach. Other sections include news from programs implementing the model, 
international applications, current and projected training opportunities, instructional tips from 
classroom teachers, and other articles with practical applications for those interested in 
implementing this model. (Copies of the newsletters are included at the end of this report.) 

Among the requested print materials are our two project monographs. The first, a 73-page 
monograph, describes the 25-year history of the Developmental Therapy-Teaching model. This 
product was useful for potential sites which needed in-depth information about development of the 
model, references for related publications, summary of research conducted in a variety of settings 




12 



documenting student progress with this model, acquisition of skills by teachers, and adaptations of 
the model for inclusive and early childhood settings. The second monograph, a 33-page publication, 
Documenting Effectiveness, summarizes research evidence of model effectiveness at five locations 
with children in inclusive, partial inclusion, or special education settings. Additionally, observational 
data ofteaching teams and college interns rated on performance in demonstrating specified practices 
for model implementation when working directly with children are included. Over 200 monographs 
were distributed. 

Our web site began operation in August, 1998. We received an estimated 7,500 inquiries 
(approximately 300 hits a month), some being direct inquires about how to obtain additional 
information about the model. These requests were from parents, teachers, administrators, and 
graduate students about individual training in the use of this model. We also received requests from 
professors for more information about the model, and from program administrators seeking 
workshop training and outreach assistance for staff development, providing new services or 
improving existing services to this age group. 

In addition to disseminating directly through our office and web site, awareness materials 
about the model are included in several resource directories, including: 

• CL AS Culturally & Linguistically Appropriate Services. Early Childhood Research Institute, 
OSEP/USDE, UIUC-ERIC, CEC-ERIC., 1999. 

• EEPCD Resources for Autism Spectrum Disorders. Compiled by NEC*TAS, 1998. 

• EEPCD Resources Supporting Inclusion. Compiled by NEC*TAS, 1998. 

• EROD, Education Resource Organizations Directory . U. S. Department ofEducation,1998. 

• ERIC, 1998. 

• ISER Home, Internet Special Education Resources, Special Education & Learning 
Disabilities Resources: A Nationwide Directory. Feb. 1999. 

• Map to Inclusive Child Care Network publication, 1998. 

• Tests in Print. Complied by the Buros Institute of Mental Measurements, University of 
Nebraska, 1999. 

Dissemination activities have also been conducted through presentations at professional 
meetings, conference exhibits, workshops, and papers prepared for publication. These activities are 
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reported separately in Management Objective 5, Professional Development. 



Management Objective 2, Site planning for model implementation: Planning to identify and 
design outreach services which reflect the individual needs of potential participants and programs. 

This management objective focused on planning outreach services that met the specific needs 
of programs requesting assistance. We call this activity site development — Phase Two in our 
original outreach design. Following a preliminary request for outreach assistance, the planning focus 
was to identify the training needs at local program sites and to assess the degree of commitment of 
staff, parents, and administrators to provide necessary resources and support for model replication. 
Criteria for accepting a site for model replication were: 

/ Evidence of administrative support and need for services to be provided for model 
implementation. 

/ Evidence of sufficient staff planning to demonstrate basic knowledge about the 
model and a willingness to attempt model implementation. 

/ One supervisory person from the site who agreed to participate in the Training 
Trainers Program while outreach services were provided to the direct service 
personnel. (New requirement, begun in Project Year 2.) 

/ A Training Agreement with content needs and training schedule collaboratively 
developed from a needs assessment developed by project staff, program 
administrators, direct service providers, and parents. 

/ A student evaluation schedule for the year, including a minimum of pre- post- 
measures to be submitted to the project without student names attached, including 
the Developmental Teaching Objectives Rating Form-Revised (DTORF-R) and other 
evaluation measures routinely used by the program. (See Management Objective 8, 

Evaluation.) 

/ Agreement to use the DTORF-R for IEP program planning and to provide family 
services and program evaluation consistent with the principles of the model. 

/ Commitment of staff time to training with the understanding that periodic 
performance measures would be collected by the project instructor with feedback to 
the participants. 
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The planning phase with a program was completed when a formal training agreement was 
negotiated and signed by the site program administrator and the project director. This agreement 
formalized mutual agreements about (a) amount of training and technical assistance the project 
would provide, (b) obligations of the local site for cost-sharing and released staff time, and (c) 
scheduled times for repeated evaluations of children's progress during the training period. Each 
training agreement specified the implementation sequence but could be modified during the process 
as needed. 

When a local program committed to model implementation, cost sharing was negotiated on 
the basis of the size of the program, the number of participating teams, extent of administrative 
support, and the initial skill levels of participants. Local programs were expected to contribute some 
resources to the effort. We used a minimum cost-sharing approach in which every participating 
program made some degree of commitment, both in cost and in released time for participating 
personnel. The fees were designed to minimize initial costs to local programs during each planning 
phase and included only minor training and travel expenses for the project consultant/trainer. The 
project assumed major costs for training materials and the project consultant. In contrast, there was 
a rather large proportional contribution expected when a program simply requested a single 
workshop. This type of assistance was kept to a minimum, except for introductory presentations 
about the model. Workshops alone, without follow-up, seem to have few long term benefits; i.e., 
there is little carryover or skilled implementation. (A copy of the fee schedule is included in 
Appendix B the end of this report.) 

Figure 2 lists the sites participating in planning for model implementation and shows the 
year a site initiated planning and outreach activities as well as the extent of carryover of training 
from year-to-year. As shown, 20 sites received planning assistance for model implementation. 
Overall 20 programs participated in project activities. Nine of these programs were solely early 
childhood programs serving children through age 6. Of the remaining 1 1 programs, 5 served both 
preschool and school aged children with Autism Spectrum Disorders or severe developmental delay 
and 6 served a wider range of ages including those in early childhood. Thirteen sites were identified 
as inclusive programs. At 1 3 sites, 20 personnel in supervisory positions participated in leadership 
training activities (sites identified with an asterisk* in Figure 2). 

Each program differed in staff skills, needs, and resources. Planning for the training 
sequences and implementation processes at each program site was unique to the identified needs. 
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Therefore, considerable resources were allocated to the planning phase of outreach, prior to actual 
training for model implementation. Additionally, a new training agreement and a revised training 
plan was re-negotiated at the end of each project/school year if outreach assistance continued. As 
local service providers became increasingly proficient, the performance data generated during the 
period of outreach assistance indicated strengths and weaknesses in proficiencies. These data were 
used in collaborative planning between the administrators, participants, and our project instructors 
during each phase of outreach. 

Management Objective 3, Model Implementation and Replication: To provide outreach 
assistance for model replication in general education and natural settings whenever possible. 

This management objective received the major portion of project resources and staff time 
because activities involved extended, in-depth training with repeated visits to each participating 
team at each site. Overall, during the three project years, activities to implement the model provided 
224 days of direct, on-site consultation and instruction through inservice, observations, feedback, 
and tutorials at 20 program sites with 346 participants working directly with 585 children with 
special needs. Original project outcomes from these implementation activities were projected to be 
between 14 and 18 local sites implementing the model with approximately 200 participating 
individuals. 

The numbers participating includes parents and program support staff who were in positions 
to participate for extended periods of in-depth training; e.g., parents, social workers, program 
directors/principals, psychologists and general education teachers. However, the numbers do not 
reflect the parents and additional local personnel that participated in introductory workshops and 
staff debriefings only (These individuals are described in Management Objective 5, Professional 
Development.) 

Typically, a site seeking implementation assistance received an initial visit for observation 
of the program, a preliminary needs assessment, and an overview presentation to staff about the 
model. In programs where it was clear that there was commitment to model implementation, this 
first visit/observation often offered an opportunity to make a pre-training performance assessment 
of each participating team. DT/RITS observational ratings of the teams and an Administrative 
Support Checklist were used to obtain baseline information, providing invaluable information about 
training needs as described in Appendix H of the original proposal. 
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The type and extent of outreach services was determined by the annual re-assessment of 
needs and the interim performance data at each site. Sites with an outstanding program, documented 
proficiency, and meeting the standards set for the replication standard were encouraged to attempt 
to achieve the highest demonstration standard during the last project year. This standard indicates 
that site personnel are sufficiently skilled to demonstrate model practices consistently as an 
exemplary replication which can be used for observation and training by others. Three sites reported 
in Figure 2 were able to achieve this high quality standard by the end of the third project year, and 
will become future training sites where others may observe exemplary practices and receive 
guidance for implementing the model practices in their own programs (See Appendix C for 
replication standards). 

Each of the 20 sites which received outreach in the basic model implementation during this 
project period had a unique approach to service delivery and varying amounts of parental 
involvement. Figure 3 provides summary characteristics of 20 sites receiving in-depth staff training. 
The Early Childhood Special Education program of the Cooperative Education Services in Trumbull, 
Connecticut provides an intensive therapeutic program, using a general preschool curriculum with 
highly individualized adaptations to provide for the special factors needed by their young children 
who have severe language, hearing, behavioral, and autistic disabilities. The Gateway-Longview 
Therapeutic Preschool in Bowmansville, New York is a therapeutic day care program for children 
with and without disabilities. During our assistance, they expanded their pullout program for those 
with special needs into their general preschool curriculum program for nondisabled peers. The 
Special Needs Program, part of the Positive Education Program in Cleveland, Ohio, and Sunshine 
& Rainbows, Forks, Washington, offer special, inclusive and community programs for individuals 
with severe developmental delays and the autism spectrum. 

The Monarch Therapeutic Child Care Program in Lacy, Washington offers highly specialized 
child care and preschool curriculum to young abused children who are under state protection through 
the Therapeutic Child Development (TCD) Program in the Department of Health and Human 
Services. Parents and foster parents participate actively in training activities. Three additional 
Washington State Therapeutic Child Development programs in Grayson, Sunnyside, and Yakima 
offer similar programming. The Learning Tree Preschool Program in Bremerton, Washington focuses 
primarily on preschool children but also provides before and after school programs for older children. 

The Early Intervention Program, a component of the Positive Education Program in 
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Cleveland, Ohio, is jointly funded by public school special education, Medicaid, community sources, 
and United Way. The parent-guided, community-sponsored service is for parents, grandparents, and 
foster parents of very young preschool children who are having difficult behavior 
management/discipline problems. Parents who have "graduated" from parent services work under 
EIC staff supervision with other parents conducting initial interviews, coaching in effective 
management strategies, and doing follow-up. Developmental Therapy - Teaching outreach activities 
for staff and parents guided a shift in service delivery to provide a fuller inclusive early childhood 
curriculum to include normally developing preschool siblings. 

The Audubon Area Preschool program within the Hopkins County, Kentucky school district 
incorporates a wide service area with many inclusive early childhood programs. A second 
geographically extensive program in Hopkins County serving kindergarten children in inclusive 
settings participated in outreach assistance activities. Additionally, during Year Three, outreach 
assistance for a third Hopkins County program for school age children with severe disabilities (ages 
5 years - 14 years) previously served through our severe outreach project was continued. Our specific 
implementation activities for this project were directed to the children eight and under at these sites. 

Robins Air Force Base schools in Georgia had two projects to implement Developmental 
Therapy-Teaching; one, an inclusive program with young troubled children in first and second 
grades; and the second, a program which served children with developmental delay or autism. There 
was strong parent participation in outreach activities. Also, in Georgia, three psychoeducational 
programs serving children in self-contained and inclusive programs received outreach services. 

Each of these sites has performance data for progress of children and proficiency of staff. 
Statistical analysis indicates that during this project period 228 children at 13 sites made significant 
gains in social-emotional-behavioral development, as measured by the DTORF-R (see Management 
Objective 8). Annually, data provided the basis for re-assessment of training needs, amount of 
outreach assistance provided, and topics covered. Additional sites were served during the last project 
year because of the availability of local trainers as they became certified as Regional Associates 
(RAs) for Developmental Therapy-Teaching (see Management Objective 7). 

Management Objective 4, Interagency Collaboration for Model Implementation: To coordinate 
outreach activities with state and national agencies to improve services for young children with 
social-emotional-behavioral disabilities. 
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The project collaborated with 7 state agencies responsible for personnel development to 
ensure consistency with personnel standards and plans. Over the project period, contact was made 
with educational agencies and service provider systems in Georgia, Ohio, Kentucky, Maine, 
Connecticut, New York, and Washington State. In Georgia, we participated in initial planning with 
a newly forming Early Childhood Network under the leadership of Georgia Public Television, 
Educational Division, and R*TEC for the Southeastern States and Virgin Islands Region. Project staff 
also participated in the state-wide autism planning initiative through the Babies Can’t Wait project. 
Close involvement with Outreach staff of the University of Georgia College of Family and Consumer 
Sciences provided training opportunities across the state. We joined Cooperative Extension in 
presenting statewide training for early childhood service providers. Courses in Developmental 
Therapy-Developmental Teaching assessment and methods currently under way will be offered as 
components of a new online University of Georgia continuing education program for Child 
Development Associates. In depth training with several Georgia school districts and 
psychoeducational programs led to a collaboration with the Georgia Department of Education on a 
State Improvement Grant. Planning is underway with the Georgia Department of Human Resources 
for intensive training for foster families (Department of Family and Children Services) and technical 
assistance for therapeutic childcare programs (Mental Health, Mental Retardation and Substance 
Abuse). 

In Ohio, training during Project Year Three continued with Positive Education Programs, a 
broad umbrella organization serving children with special needs from 200 school districts in the 
metro Cleveland area. A major component of our work in Ohio has been preparing six leadership 
personnel to independently provide staff development to expand model implementation to programs 
both within PEP and in the geographical area. Three program specialists/coordinators have 
completed certification as Regional Associate Instructors. By providing technical assistance in 
Developmental Therapy - Teaching methods and assessment to Cleveland area schools thrbugh the 
Positive Education Programs, Regional Associates insure sustainability of the model. 

Through planning with the State of Washington, we completed an interagency agreement with 
the Department of Social and Health Services, Mental Health Division, the Children's Administration, 
and the Washington State Educational Service District 113. This agreement enabled our project to 
provide leadership training to program coordinators of special programs throughout the state. Results 
of this collaboration and the RA training-trainers program are described more fully in Management 
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Objective 7, The Regional Associates Certification Program. 

Outreach to early childhood programs in several other states provided educational and mental 
health services in a variety of settings. The effectiveness of our training is due to the successful 
collaboration and joint vision of many state and regional agencies including the Head Start program 
of Audubon Area Community Services, Inc., (Kentucky), Gateway Youth and Family Services, (New 
York), EASTCONN, (Connecticut), and Maine State Billing Services, Inc., (Maine). Affiliation with 
state and national professional associations and agencies such as Georgia Association on Young 
Children, CEC, CCBD, DEC, NEC*TAS, National Research Institute on Children's Mental Health, 
The Federation of Families for Children's Mental Health, Center for School Mental Health 
Assistance, et al, provided opportunities for networking which are and will be invaluable to the health 
and growth of Developmental Therapy - Teaching Programs and the positive impact on the lives of 
special children. 

During the three year period, the project activities were approved by 8 universities in 5 states 
for staff development units (SDUs). The agencies for licensing child care providers approved the 
Developmental Therapy-Teaching Programs training under licensure standards in the State of 
Washington and in Georgia. Additionally, Continuing Education Units (CEU's) were provided to 
participants in the Black Hills Seminars in South Dakota through Augustana College; in Wisconsin 
through Silver Lake College Spring Tonic Conference; in Texas at the International CCBD 
Conference through University ofNorth Texas; and in Georgia through Georgia State University and 
the University of Georgia Center for Continuing Education. Developmental Therapy - Teaching 
comprised instructional units in undergraduate and graduate classes at the University of Georgia, 
Georgia College, University of Wisconsin at Eau Claire, and the University of Southern Maine. 

Activities for continuing multi-agency collaboration during the final project year focused at 
the national level on cooperation with groups advocating community responsiveness to mental health 
needs of young children and to parent groups seeking expanded, exemplary services. We cbntinued 
the linkages that we had built during the first two project years. These activities have had significant 
impact on the expansion and improvement of our project’s outreach efforts and on services to very 
young children, especially long term benefits for increasing the scope and effectiveness of local 
services. 
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Management Objective 5, Professional Development: To provide professional development 
through workshops, presentations at professional meetings, distance learning, and electronically 
mediated communications (EMC). 

The project provided topical workshops, on request, for professional development (inservice), 
presentations at local, state, and national professional gatherings, and extended trainings which 
included direct tutorial assistance in classrooms. Professional development activities encompassed 
two categories: (1 interactive workshops in which professionals, paraprofessionals and parents 
frequently participated together and (2) conference presentations, papers, and training at professional 
meetings/seminars. Our experience demonstrated that extended, in-depth training offered greater 
potential than single workshop formats for professional growth; therefore training in this format was 
not a project priority. However, single workshops and other forms of discrete training remained in 
demand and provided an excellent vehicle for introductory presentations of the basic approach and 
its components to a variety of audiences. 

Over the three year period, 1,650 individuals attended workshops/conference presentations 
about Developmental Therapy-Developmental Teaching topics. (This count reflects repeated 
attendance if individuals attended more that one workshop/conference session.) Attendees of 
professional development activities represented a wide range of professional occupations and 
personal experiences. Figures 4a (Project Years 1 and 2) and 4b (Project Year 3) summarize the 
occupations of these workshop/presentation respondents. Combining the numbers qf participants in 
each category during the three-year period revealed the following occupation information: 38% 
teachers, 17% paraprofessionals, 1 1% mental health professionals, 1 1% administrators/supervisors, 
3% social service providers, 3% health care professionals, and 5% parents. 

While many of these training activities were targeted solely to model applications for early 
childhood, other presentations and discussions were extended to children’s social-emotional health 
issues across the age spectrum. The topics most often included model overview and fts basic 
components: Assessment of Children’s Social-Emotional-Behavioral Development, Functional 
Behavioral Assessments, Developmental^ Appropriate Behavior Management, Anxiety and Defense 
Mechanisms, Curriculum Practices that Address Emotional Needs, Effective Classrooms and 
Schedules, Building a Therapeutic Team. 

Topics of a more theoretical nature were presented at professional meetings e.g., Emotionally 
Healthy Practices; Developmental Therapy-Teaching as a Framework for Contemporary 





Psychoeducation; and Agency Collaboration for School-based Mental Health services for Children. 

Conference presentations, seminars at professional meetings and published papers are among 
the proven, traditional means of promoting professional development that have been used by this 
project as outreach vehicles. Sample presentations, papers, and exhibits by project staff listed in 
Figure 5 show the wide range of topics delivered and audiences reached during this project period. 
The project provided an all-day preconference training session in Developmental Therapy-Teaching 
at the Infant and Early Childhood Annual Conference in Olympia, Washington, in May, 1999 and 
a four-session strand on psychoeducational methodology at the International CCBD Conference in 
Texas in October, 1999. National Instructors and Regional Associates conducted all-day 
preconference sessions as well as several stand-alone sessions at the Black Hills Seminars in South 
Dakota in June 1998, 1999, and 2000. National Instructors and Regional Associates also presented 
a four-day intensive training for the 2000 Connections Academy in Wenatchee, Washington in July 
and a three-session strand on Developmental Therapy-Teaching at the 2000 Georgia 
Psychoeducational Network Conference in August. 

Professional presentations, evaluated for effectiveness, is a required standard for Regional 
Associate (RA) certification. During the three years of this project, a growing number of RAs co- 
presented with National Instructors or led workshops, seminars, and conference presentations, 
resulting in a dramatic increase in the actual numbers of workshops/presentations during Project Year 
3. To date, 100% of the Early Childhood RAs have co-presented at regional or national trainings; 
65% have achieved competency on this certification standard. RAs had an integral part in over half 
of the workshops/presentations in Year 3 . 3 1 % of these were co-presentations by RAs with National 
Instructors and 30% were independently initiated by RAs (See Management Objectives 7 and 8). 

The demand for quality professional development activities has highlighted the need for 
alternative delivery modes which are manageable by a small staff. An effort has been made to use 

4 

electronically mediated communication for training, e.g., internet, and two-way videoconferencing, 
whenever feasible. To date, personal interface is still preferred by both staff and target audiences; 
the face-to-face interaction between trainers and trainees is still an essential component of 
professional development activities. However, as technology improves and becomes more widely 
accessible and our ability to utilize it effectively increases, electronically mediated communications 
is expanding our outreach efforts to reach a larger audience in a cost effective way. 
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Management Objective 6, Product Development: To develop or revise instructional products and 
media packages for use in outreach training. 

In order to reflect current needs and trends of the field, project staff worked continually to 
develop, disseminate, evaluate, and redesign the extensive products, materials, and instructional 
modules used in this outreach project. New materials/products included (a) new video productions 
for introduction to the model, (b) computer-aided materials for meeting 1997 IDEA requirements for 
functional behavioral assessments, (c) new training materials for trainers-in-training, and (d) new 
training materials for skill practice by site personnel. These products are described below: 

New video productions . In addition to new printed awareness materials and portfolios of 
model information, the project completed a series of introductory 20-minute videos in collaboration 
with the University of Georgia Center for Continuing Education, Media Production Department. The 
videos introduced principles of the Developmental Therapy-Developmental Teaching approach and 
illustrated exemplary practices to demonstrate developmental^ appropriate strategies and 
environments which encourage social-emotional growth. These videos were designed for personnel 
and families of children in three age groups: (a) Early Childhood and ECSE programs, (b) elementary 
school programs, and (c) middle and high school programs. The first, “Providing Developmental 
Therapy-Teaching Programs for Little Ones,” was completed in March, 1998. The second, 
“Developmental Therapy-Developmental Teaching for Troubled Children in Elementary School,” 
was completed in August, 1999. The third was re-designed into an interactive simulation CD-ROM 
format. Additional funding was needed to accomplish this and was received from the U. S. 
Department of Education, Office of Special Education Programs. This project is currently underway 
with separate funding. A fourth proposed video for training personnel to use the Developmental 
Therapy-Teaching Objectives and Rating Form-Revised (DTORF-R) with reliability was also re- 
designed into newer technology using an internet course format. 

Computer-aided materials. The software version of the DTORF-R for use by participating 
sites was completed in 1998. This program, available for either MAC or PC, enables a rating team 
to generate their results in printed format to attach to their local IEP forms. In response to the 1997 
IDEA requirements for identifying specific social-emotional-behavioral objectives, this software 
provides a section for the IEP on social-emotional-behavioral objectives and criteria for mastery, 
along with a functional behavioral assessment and recommendations for positive behavioral 
interventions. The software also allows for repeated measures to be stored and a child's progress 
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record generated at repeated reporting periods. An upgraded version of this program is currently in 
progress-. 

New training materials for trainers-in-training. In keeping with the increased focus on training 
trainers (the RA leadership program described in Management Objective 7 below) we compiled core 
instructional units most frequently requested by sites. These structured “lessons” were requested by 
the leadership trainees as a means to assist them as they began to conduct inservice workshops and 
training independently. Appendix D contains the "content map," modified in Year 3, outlining the 
instructional content modules from which specific core sequences are selected. The Trainers’ Manual 
under production during the latter half of the last project year, has been distributed on a pilot basis 
to each trainer as they completed requirements and met performance standards for certification as a 
Regional Associate (RA). This activity was sponsored jointly between our two outreach projects, 
sharing costs and staff time. 

New training materials for skill practice bv site personnel. In addition to redesigning handouts 
and participant exercises to support existing training workshop modules, we began to reshape this 
management objective to focus on distance learning and web-based communication. A strategic plan 
was developed for the expanded use of technology and distance learning through teleconferencing, 
consultation and training via the University satellite system, and the internet. (See Appendix E). 

Considerable planning effort went into exploring the feasibility of designing CD-ROM 
interactive programs for assisting practitioners in acquiring specific positive behavior management 
skills. This plan was described in Management Objective 6 of our Year One performance 
report/refunding proposal. The conclusion was reached that it is a critical need, but beyond the scope 
of this project's resources. Therefore, we prepared two grant applications for this specific product 
development. One effort for elementary age children, was directed for inclusion in Georgia’s State 

Improvement Grant application, and the other, for preschool and teens to the Office of Special 
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Education, Projects ofNational Significance. Funding was received in both instances and the projects 
are in process. Efforts are cdntinuing to obtain additional funding for further development of new 
technology — specifically interactive internet course work and CD-ROM instruction, offering useful 
tools for adults to learn and practice at their own learning rate. 

Management Objective 7, The Regional Associates (RA) Certification Program (Management 
Objective 8 in Year 2 Performance Report): To design new outreach activities and modify existing 
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strategies to accommodate the needs of personnel in varied settings. 

In the original proposal, it was anticipated that activities in this management objective would 
focus on the design and production of innovative, new or modified instructional activities to enhance 
participants’ understanding and mastery of model practices in the many and diverse settings where 
the model is being replicated. After the first project year, however, it became clear that long-term 
project impact on participating programs must rely on local leadership to provide needed support for 
trained personnel, re-train new staff, maintain program integrity, and document the gains made by 
children served. In response to this long-range need, the project initiated a pilot "training-trainers" 
program for local leadership people at participating implementation sites to prepare them to (a) 
conduct awareness sessions and basic inservice training for new staff to use the basic model 
components, (b) guide their experienced staff in maintaining high quality performance, and (c) assist 
new local programs in planning and implementing the model. 

Criteria for acceptance into the RA Training- Trainers Program were: 

^ Hold a current supervisory/coordinating position with responsibilities for direct 
supervision in a program planning to implement the model. 

^ Submit a resume of prior experience related to work with children and teens who have 
severe social-emotional-behavioral disabilities. 

^ Provide a letter of recommendation about current work from a supervisor. 

^ Complete a preliminary needs assessment. 

^ Complete a pre-assessment test of basic knowledge about the model and how its 
theory is translated into "best practices" for the students to be served. 

^ Commit to follow-up with model outreach activities independently for at least one 
new site a year following certification. 

Those accepted into the RA Training - Trainers Program planned their individual training 
programs with a project instructor, so that their training occurred simultaneously with co- 
teaching/supervision of local program personnel. Training was designed to be completed in two 
years. 

First year training domponents for participating RAs focused on acquiring knowledge about 
Developmental Therapy - Teaching: 

/ Complete independent study modules about the basics of the Developmental Therapy- 

Teaching model and the underlying principles. 
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/ Participate and co-teach with the project instructor for on-site workshops. 

/ Observe with the project instructor as direct service teams learn to implement the 
model in their work with students. 

/ Complete three observational ratings of the teams with the instructor: a baseline rating 

at the first observation and two practice ratings during the second & third 
observations. 

/ Debrief with the instructor and the observed team following each observation to 
provide feedback about ways model practices were demonstrated and ways 
performance could be improved. 

Second year RA training focused on applying Developmental Therapy - Teaching knowledge 
and skills under the supervision of a National Instructor : 

/ Plan and implement (with instructor's assistance) introductory workshops for new 
staff members and additional workshops on essential model elements for all returning 
personnel. 

/ Observe teams with the project instructor during all site visits. 

/ Complete three observational ratings with the instructor: two practice ratings with 
feedback (first & second visits) and one "reliability" rating to measure degree of 
agreement with the instructor, item-by-item. 

/ Debrief with the project instructor and teams after each observation about ways they 
are demonstrating model practices were demonstrated and ways performance could 
be improved. 

/ Assist the project instructor in preparing written descriptions of workshop activities 
proven effective in staff development for implementing the model. 

Twenty early childhood leadership level individuals in six states actively participated in the 
RA program during this project period. Appendix F contains the names and positions of these, 
trainees. The competencies and evaluation sources for leadership participants specified as 
requirements for certification are summarized in Appendix G. Although the training is offered as a 
two-year sequence, a considerable amount of the leadership training experience must be done around 
daily job responsibilities, making it difficult for some to complete these requirements in a timely 
manner. Not all were expected to complete this pilot training during the project funding period, and 
other funding sources were obtained to enable these to continue their training. At the end of the final 

O 

ERIC 



18 



26 



project year, 6 participants had completed all certification requirements, and 1 2 others were actively 
nearing completion of the requirements. Details of individuals' progress toward meeting required 
standards is found in Management Objective 8, Evaluation. 

Training was comprised of a variety of experiences. We provided individual tutorials, 
planning consultation, workshop co-teaching opportunities, and observation/feedback to Regional 
Associate trainees about their presentation and supervisory skills. In some regions, individuals met 
informally in small groups each month, usually after work or on Saturdays, to share their experiences 
in implementing the model; some held quarterly tutorial phone conferences with their national 
instructors; many observed in each others' respective programs for practice with the instruments. 
(Materials describing the Regional Associate requirements, performance standards, and evaluation 
plan were provided in Year One Performance Report/Refunding Proposal.) Articles describing the 
activities of the Washington State Regional Associates and the Ohio State Regional Associates are 
included in the newsletters (see enclosed newsletters at the end of this report). 

The project sponsored an invitational conference for Regional Associate leadership trainees 
in April, 1998. This was attended by 13 of the early childhood leadership Regional Associates in 
addition to 1 1 other participants from school age programs. This conference provided 20 contact 
hours of instruction in model implementation, theory, and practice. Extending over three and a half 
days, the conference was well-received by participants. The summary of participants' evaluation of 
the conference is included in Management Objective 8, Evaluation. 

Two interactive teleconference sessions were held for RAs to share their leadership 
experiences and to help us reassess our goals and activities. As our numbers of certified instructors 
trained under the auspices of this project and companion projects increase, a newly forming Regional 
Associate Network is becoming more active. We envision that this Network, by maintaining close 
communication with project staff, will enable training to reach a larger audience without sacrificing 
model fidelity. The Network members will also benefit from the ability to share ideas and resources 
with Regional Associate Instructors in other geographical locations. 

As reported previously in Management Objective 5, these trainers-in-training also participated 
as co-teachers/facilitators with project staff instructors in workshops or other professional 
presentations. The workshop supervised by the project instructors and were evaluated by participants 
and project instructors. This provided feedback about the knowledge the RA's had acquired about the 
model, their skills in facilitating involvement of workshop participants, and their overall effectiveness 





as workshop presenters. In addition to their contributions in maintaining the integrity of sites 
replicating this model, certified Regional Associates have been called upon by other local programs 
to assist in implementing the model. 

Management Objective 8. Evaluation (Management Objective 7 in Year 2 Performance Report): 
To evaluate project effectiveness in meeting the original project goals on time and within budget. 

Accomplishments for each management objective were evaluated for timeliness and 
effectiveness. Figure 1, presented at the beginning of this report, summarizes the scope of project 
accomplishments. Forms and instruments have been developed and field-tested in previous projects. 
They were included in the original proposal with descriptions of their development, reliability, 
validity, and uses. However, the Summary of Evaluation Plan from the original proposal, is 
reproduced in Appendix H for ease of reference. It is particularly important to note that the original 
reliability and validity studies about these instruments and subsequent research reports using these 
instruments were submitted to U.S. Department of Education, Program Effectiveness Panel. These 
studies were conducted with populations of students with identified social, emotional and behavioral 
disabilities. The performance measures in these studies resulted in documentation of program 
effectiveness and model validation three times - first, as an effective model for children with severe 
social-emotional-behavioral disabilities; second, as an effective training program to increase 
performance competencies of those who work with these students; and third, as an effective model 
.for use in. inclusive, partial,, opspecia} education settings. 

Reliable data collection at ongoing service program sites is a well-documented challenge. 
It proved to be so for this project as well. In order to assure confidence in reliability of the data and 
in the accuracy of the findings, we had to accept smaller numbers in our samples. While this approach 
introduced a question of bias into the selection process, we chose samples which had reliable data, 
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excluding those where data were incomplete or inaccurately collected. We believe the smaller 
samples are representative of the typical participants, children served, sites, and outcomes. 

Effectiveness of this project was assessed on four dimensions: (a) observational measures of 
participants’ performance in using the specified practices; (b) progress of the children served by the 
participants during model implementation; (c) satisfaction of the participants with the training; and 
(d) assessment of administrative support for model implementation. Results are presented below. 
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Evaluation question 1 . Do participants demonstrate understanding of the, social-emotional- 
behavioral development of children and their own roles in fostering healthy social-emotional- 
behavioral development of children and youth? 

The original evaluation plan involved administering a series of quizzes in multiple choice 
format to answer this question. A decision was made during the first year of the project that a 
knowledge test per se was not entirely suitable because of the intense focus of our technical 
assistance on actual performance and demonstrated skills of participants. The proposed quizzes put 
some direct service participants at a distinct disadvantage and resulted in low scores when their actual 
performance demonstrated understanding of the content. We believe that effective performance 
requires understanding of the knowledge base. Therefore, observations of performance using the 
Developmental Therapy-Teaching Rating Inventory of Teacher Skills (DTRITS) was accepted as a 
sufficient proxy of knowledge on the part of the direct service teams. A representative sample of 
participants performance scores when working directly with children is presented below in evaluation 
question 2. These results show that 84% achieved DTRITS proficiency scores of adequate or better, 
indicating a working knowledge of basic practices needed for model implementation. 

To further evaluate participants understanding of their roles in model implementation, a group 
of 13 program directors and coordinators volunteered via teleconferencing to participate in a focus 
group to discuss their views of the training program. The in-depth, open-end interview was conducted 
by the Project Evaluator and the Coordinator for Distance Learning in a one-hour, informal 
conversational format using video tape to record responses. Focus group guidelines suggested by M. 
Q. Patton in How to Use Qualitative Methods in Evaluation were followed for conducting the 
interview and analyzing responses. Here is a quote from one participant in the focus group that 
reflects his/her views about understanding important elements in successful implementation: 

7 think my experience is that I am able to walk into a classroom, 
whether it is a classroom like the preschool classes at our center. ..I 
am able to walk in there and tell the teachers what is going on with 
/child/ dynamics and I am able to support their [the teachers'/ 
strengths and tap in and understand appropriate expectations for that 
child... and what anxieties are constantly at play for kids. Also, I am 
able to walk into other classrooms and other settings where a parent 
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may be involved and without seeing a child for a tremendous amount 
of time be able to facilitate the assessment process. And be a /model/ 
teacher for parents and teachers about appropriate expectations and 
really target plans that are going to help support a child's growth. 



This quote is representative of feedback we received repeatedly as we worked across sites and 
reinforces our decision to look at actual performances as a proxy for knowledge and understanding. 

Evaluation question 2. Do participants demonstrate effective performance skills in the 
service setting after participation in the training program to implement the model? 

Performance of direct service participants. Model implementation requires major emphasis 
on close teamwork among the direct service providers and the support/resource staff. When project 
staff made site visits for in-depth follow-up training, an attempt was made to observe all 
participating teams for a minimum of one hour in each classroom. Typically, an observation was 
then followed by a 30 - 60 minute debriefing for feedback with the team, focusing on skills and areas 
of performance that required improvement. At the time implementation activities began at a site, the 
project instructor observed each team to obtain a baseline DTRITS rating (Time 1). After the initial 
implementation activities were completed, DTRJTS ratings were repeated (Time 2). This procedure 
was repeated each year that the site participated with the project in implementation. 

Table 1 reports the DTRITS scores achieved by 45 teams (97 individuals) at 13 
representative sites after initial model implementation, performance feedback, and tutorial 
assistance. Levels of proficiency established for DTRITS scores in previous studies are 90-100 =' 
Highly Effective, 70-89 = Effective, 50 - 69 = Adequate, 30-49 = Less than Adequate, and 16 - 29 
= Poor. The scores indicate that 76% of these teams (64 individuals) achieved DTRITS proficiency 
scores at the Adequate or better level, indicating demonstration of the basic practices necessary for 
model implementation. Of these teams, 80% (58 individuals) demonstrated Effective or Highly 
Effective skills. 

Performance of leadership participants in the training-trainers pilot program . An expanded 
evaluation design was added in Project Year 1 when the new pilot program was initiated for early 
childhood leadership individuals in the Regional Associate (RA) Training Program. Table 2 lists 
the progress of these participants toward achievement of the trainers' standards. Results of the 
evaluation activities for this pilot training-trainers program component are summarized here: 
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Competency 1. Knowledge : The 100-item multiple-choice test of knowledge about 
Developmental Therapy-Developmental Teaching was taken by all but two of these RA leadership 
participants at the beginning of their training. Post training knowledge tests were administered on 
an individual basis, when each participant requested the test after periods of self-study or as they 
came to the end of their individualized leadership training program. Nine of the twenty trainees took 
and achieved the passing criterion or greater, with an average gain of 15 points. Table 3 reports the 
pre-and post-training scores for these individuals. The remaining participants are continuing their 
independent progress toward certification as a Developmental Therapy-Developmental Teaching 
Regional Associate through another grant project which resulted from this pilot effort. 

Competency 2. Reliability in using the 171-item DTORF-R assessment procedure : 
Leadership participants (RAs) were expected to participate in team assessments of children in their 
programs and to review all DTORF-R ratings for accuracy. This procedures is a quality check on 
reliability of the assessment and requires extra proficiency in the use of the instrument on the part 
of the RA. Each rating was then reviewed by a project instructor for accuracy in the rating procedure 
and reliability of rater judgments. The instructor identified problem areas or inaccuracies in the 
rating procedure and provided feedback to the RA and the rating team. When DTORF-R ratings at 
a site were accepted as reliable and valid measures by the instructor, the RA was judged to have 
passed competency 2, DTORF-R reliability. Using this procedure, to date 13 RAs received a “pass”, 
indicating competency in supervising team ratings of social-emotional-behavioral development. 

Competency 3. Reliability in using the 212-item DTR1TS observational rating form\ RAs 
were expected to observe with the project instructor as teaching teams worked directly with groups 
of children during implementation of model practices. These parallel observations were made during 
each return visit of the project instructor, and practice DTRITS ratings were made independently 
by the RA and the instructor. Follow-up discussion of rating differences on particular items 
following an observation served as tutorials for the RAs. This procedure was repeated with ehch visit 
until the DTRITS rating by the RA reached 80% agreement with the project instructor. Using this 
procedure, 14 RAs achieved the performance criterion to date. 

Competency 4. Field supervision: Each RA was expected to provide on-going inservice 
assistance to their staff for model implementation during the periods between project instructors’ 
visits. At the conclusion of the training agreement, or at the time when the RA and project instructor 
believed that implementation reached an acceptable replication level, the teams were asked to 
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anonymously rate the quality and effectiveness of the RA in assisting them in effective 
implementation. Using this procedure, 7 RAs completed the requirement successfully by receiving 
average ratings of 4 or better in field supervision on an 8-item form with a 5-point rating scale (See 
Appendix I). As this outreach project ends, 8 other RAs are actively in process of guiding their 
program staff in model implementation and 5 are inactive but indicate interest in continuing to use 
model components. 

Competency 5. Group instruction in basic model elements : Three phases of training were 
used to assist the RAs in developing effective skills for leading staff workshops for model 
implementation. The first phase, completed by 15 of the RAs, involved co-teaching with a project 
instructor in which planning was a combined effort between the instructor and RA. The second 
phase required independent presentations when there was no co-teaching but the project instructor 
assisted the RA in planning, selecting strategies, and designing effective workshop materials. This 
second phase was completed by 1 5 RAs and the presentations were also evaluated by the workshop 
participants. The third phase for certification was successfully accomplished by 13 RAs, in which 
they independently planned all aspects of the workshop, led the session, and were evaluated by a 
project instructor on an 1 8-item rating form with a 5-point scale of effectiveness as a session leader. 

Evaluation question 3. Did children show significant progress in social-emotional- 
behavioral competence during the model implementation period? 

To evaluate project impact on children served by the participants during model 
implementation, 18 sites agreed to assist us in collecting baseline descriptive data and reliable 
assessments of social-emotional-behavioral status using the DTORF-R. All children served by the 
participants at these sites were included if their baseline DTORF-R ratings for social-emotional- 
behavioral development were completed with accuracy. 

Table 4 summarizes the characteristics of these 534 children at each site at the time model 
implementation was underway. The average age for this group of children was 74 months. Some 
older children with Autism Spectrum Disorders and/or severe developmental delay served by 
participating sites are included in these analyses. Boys comprised 73% of the sample and 58% were 
Caucasian. All had at least one recorded disability, 41% with a primary diagnosis of severe 
emotional/behavioral disability, 26% with a diagnosis of Autism Spectrum Disorder, and 41% with 
additional secondary disabilities. The severity of their disabilities, calculated from the extent of 
delay on their baseline DTORF-R ratings of social-emotional-behavioral development, ranged from 







18% as severe, 45% as moderate, 38% mild, and 1 1% were in the range comparable to their age 
peers. 

Table 5 contains the mean scores and standard deviations for the DTORF-R ratings. At 
baseline, the average scores across the sites ranged from 25.33 at site 15 to 73.28 at site 8. At time 
2, the average scores at the sites ranged from 42.03 at site 1 to 80.57 at site 8. At site 1, this was a 
gain of about 6.6 items. At site 8, the gain was about 7.3 items. The average time lags from baseline 
DTORF-R ratings at Time 1 to the second DTORF-R ratings at Time 2 ranged from 3.5 months to 
9.7 months. Results of the statistical analysis of the gains using paired dependent t-tests with a 
probability level of .05 and a 2 tailed test indicate that the children at nine of the 13 sites made 
statistically significant progress in social-emotional-behavioral development (p < .05) during the 
time when their teachers received training for model implementation. 

To explore the extent to which implementation of the model may have contributed to these 
gains, comparisons were also made between the actual DTORF-R scores achieved and extrapolated 
scores assuming no intervention with this model. The prior rate of item mastery was first calculated 
by dividing the average actual baseline DTORF-R by the average chronological age. The 
extrapolated scores were then obtained by multiplying the prior rate of item mastery by the time lag 
and adding it to the baseline score. These extrapolated scores indicate what the groups would have 
achieved assuming the prior rates of mastery had continued (without intervention) during time 
equivalent to the intervention periods. Table 6 summarizes these results. From baseline to Time 2, 
87% of the children (197) at 7 of the 9 sites made significantly greater gains during model 
implementation than could have been achieved had they progressed at their previous mastery rate 
prior to implementation. These findings indicate that model implementation by the participating 
teams had a significantly positive effect in promoting increased social-emotional-behavioral 
development of the children that they served. 

Evaluation question 4. To what extent did local programs at participating sites acquire the 
basic elements for model replication? 

Model replication at local program sites An administrative checklist containing 41 basic 
program elements desirable for effective model replication was completed collaboratively by project 
instructors and site administrators/coordinators to determine the extent to which model components 
had been included in the implementation effort. If a component was rated as "provided and being 
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used consistently," the item was marked YES. If it was used inconsistently, the item was marked 
PARTIAL, and if it was not available or not implemented, it was marked NO. The total items 
marked YES provided an administrative support score for a site. Criterion levels established in 
previous research studies on model effectiveness are these: 26-41 items = sufficient number of 
components; 16-25 items = sufficient number of elements for model implementation; and 10-15 
items = essential components in place for basic model implementation. 

Table 7 reports the administrative support scores for 17 of the 20 implementation sites that 
participated in model implementation activities for the project. All of these sites were rated at the 
basic model implementation level or better. Nine sites had demonstration level components in 
place indicating the highest level of administrative support for utilizing the model effectively. Six 
sites had sufficient number of administrative elements in place for model replication. 

Evaluation question 5. To what extent are participants satisfied with their training 
experiences? 

Satisfaction with workshops. A total of 1,646 workshop participants at 116 workshop 
presentations during the three year period completed evaluations. (It should be noted that not all of 
the participants at these workshops completed the anonymous evaluation forms.) Throughout the 
project period, two evaluation forms were utilized. During Project Years 1 , 2, and 3, a four question 
form (Form A) was used; whereas in Year 3 this form was revised to a 3 question format (Form B). 
During the third year, both evaluation forms were used. Table 8 summarizes these evaluations. 
During the project period, 1,099 participants at 88 workshop/presentations responded to four 
questions form (389 responses during Years 1 and 2; 710 in Year 3). On a five point rating scale 
with 5 representing judgments such as very beneficial material, well organized workshop and 
personal needs met very well to 1 representing dissatisfaction, the respondents indicated high 
degrees of satisfaction, with average ratings ranging from 4.37 to 4.64 Consistently similar 
evaluations were obtained on the three question form during Project Year 3 from 547 respondents 
at 27 workshop/presentations. On a 5 point scale, participants rated the effectiveness of the 
workshop process, the relevancy of the workshop content, and the personal usefulness of the 
workshop process with 5 being high, to 1 low; their average responses ranged from 4.40 to 4.55. 
Respondents expressed similar levels of satisfaction with the material, workshop organization, 
general impression of the workshops, and the extent to which their individual needs were met. 
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Satisfaction of participants in training for model implementation . To obtain information 
about the level of satisfaction among those who participated in on-site training for model 
implementation, a one-page questionnaire was mailed to all participating personnel at 16 of the 18 
sites agreeing to participate in project evaluation activities. Questionnaires were given to direct 
service team members, support staff, and administrators who had attended workshops or participated 
in extended training. Using a scale from Very Helpful (5) to Not Helpful ( 1 ), they were asked to rate 
five training activities anonymously: workshops, observations in their classrooms, team debriefings 
for feedback, written feedback, and other. The questionnaire also contained 4 additional open-end 
questions about their perceptions of skills they had acquired as a result of training, positive aspects 
and weaknesses of the training they received, and changes in their effect on children and families. 

Responses were received from 82 participants. Table 9 summarizes their ratings, indicating 
levels of satisfaction ranging from 4.04 to 4.60. Their responses to the open-end questions reflected 
a wide range of individual differences in levels of training they received, from participants on direct 
service teams working year-long for model implementation to individuals such as support staff who 
had attended only the introductory staff development sessions. Responses to the question of newly 
acquired skills included understanding the social-emotional development of children better, skill in 
using the DTORF-R assessment instrument to assess social-emotional competence, using positive 
behavior management strategies effectively, working with parents and staff more effectively, and 
developing effective programs. 

Training activities they cited as strengths included all of the specific content areas in the core 
training. They reported satisfaction with the organization of these sessions, role play opportunities, 
relevant examples, small group exercises, and opportunities to discuss individual cases. They also 
mentioned knowledge levels, helpfulness, support, and skills of project instructors in providing 
practical applications as strengths in the training. 

Their reports of weaknesses of the training focused almost entirely on issues of tirfte. They 
reported that training was too short and felt a shortage in number of scheduled observations and 
feedback they received (time limitations on the part of the visiting instructor’s schedule). They also 
reflected that they would have liked more direct suggestions for activities, curriculum ideas, follow- 
up case studies, and applications in the classroom for writing goals and objectives. A few 
respondents expressed concern over adapting the model to their specific situations due to class size, 
characteristics of students, or competing demands of the academic curriculum already in place. 
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There was also a comment that the comprehensiveness of the curriculum requires in-depth work 
beyond what can be done given all the other daily requirements. 

In response to the question about positive changes in their effect on children and families as 
a result of the training, all respondents indicated YES, giving specific examples such as a common 
language among staff and families, changes in parents’ awareness of their child’s strengths and 
problems, increased ability to manage problem behavior more effectively, and increased staff team 
work. Respondents also noted that the training had given them more confidence and increased their 
feelings of success. These detailed responses are included in Appendix L. 

Satisfaction of leadership trainees (RAsT A focus group interview with 1 3 program directors 
and coordinators in the training trainers program was held at the end of Year 2 around two general 
topics with six specific questions. The first discussion focused on the training that they had received, 
and the second topic concerned their perceptions of the training they were able to provide others. 
In general, the focus group held high opinions of their individual training experiences, both 
personally and professionally. Appendix J contains the questions and summary of responses. They 
identified experiences they valued the most (questions 1 . 1 & 1 .2), citing the 3-day leadership retreat 
for in-depth immersion on the model, participating in presentations with project co-instructors, and 
observations of instructors as they provided consultation and feedback in classrooms (See Appendix 
K for evaluation results of this conference). The group identified many new skills they had acquired 
(question 1.3), including understanding of the model and how to apply it in different situations, 
working with families to support children’s healthy development, doing assessments, decoding 
feelings, and helping teachers and parents develop effective plans. 

The group was less similar in their perceptions of their own professional and personal 
experiences during training (question 1.4). Several described their self-development as highly 
satisfying and exciting, while others expressed feeling pressure to perform at levels of difficulty 
resulting in feelings of inadequacy. (One individual viewed this pressure experience as 
“unprofessionally handled.”) Their observations of their effect on children, families, staff, and 
programs (question 1.5) were all highly positive. 

Their recommendations for design of the leadership program (question 1.6) reflected 
satisfaction with effective aspects such as the notebooks of materials from others’ training efforts, 
the self-evaluation prescreening process to identify individual strengths and needs, opportunities to 
focus on aspects relevant to their daily work, and the system that responded to different learning 
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needs. 



Their recommendations included greater assistance with presentations, opportunities to 
practice presentations with peers for feedback, increased diversity of participating RAs, and 
increased time needed for preparation of presentations. They expressed some disappointment in the 
value of self-help/peer study groups where they attempted to learn from each other. 

In discussion about the training they provided others (question 2.1), they were positive and 
confident of their present level of skill for supporting others in schools, consultation and informal 
training with parents, presenting workshops and training new staff in introductory and intermediate 
levels of model implementation, using the model for Functional Behavioral Analysis and Positive 
Behavioral Intervention Plans, and informally supporting staff in consultation abqut individual 
children’s needs. They generally felt that their work with the project and with their staff (questions 
2.2 & 2.3) was well received but expressed concern that presentations offered only at the basic level 
fail to meet the needs of advanced participants. 

Their plans for training others in the future (question 2.4) included foster parent training, 
continuing on-site staff training, consultation with other school districts, and training in positive 
behavior management for general and special educators, administrators, and mental health 
personnel. They had numerous future project plans using the model. These included finding grant 
funds for expanding the scope of their program’s model implementation, using the assessment 
instruments at a statewide level, extending the model into regional school districts, and expanding 
the curriculum resources for the model. 

They all also anticipated continuing their individual tutorial programs and completing 
certification requirements so that they would be able to train others in the future. They were 
articulate about their own strengths and weaknesses and were able to suggest very specific ways in 
which the project could assist them further in gaining the skills they needed (question 2.5). They 
requested project instructors to continue visiting and monitoring their activities and programs; 
assistance in setting up grant-funded pilot programs; assistance in obtaining resource materials, 
audio-visual aids in training for model implementation, and on-line training materials. They also 
requested an annual leadership conference bringing together RAs from across the country for in- 
depth immersion in leadership issues. 
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Summary of Evaluation Results 

The preceding review of evaluation data and outcomes indicates that each of the four original 
project goals was effectively accomplished and exceeded anticipated outcomes in the original 
proposal. Timeliness was judged by on-going process evaluation activities described in each 
management objective. Overall, the project maintained the work schedule and budget for activities 
and accomplishments as anticipated in the original proposal. Changes in key personnel that resulted 
from illness and death during the first project year somewhat slowed the initial accomplishments in 
Project Year 1 . However, the targeted activities and accomplishments were recovered and exceeded 
during the remaining two years. 

Outcome measures indicate: 

♦> Increased understanding about how to promote healthy social-emotional-behavioral 
growth through exemplary teaching and behavior management practices among those 
who work and live with young children who have severe social-emotional-behavioral 
disabilities (SE/BD). 

❖ Increased skill in using adult practices proven effective in enhancing teaching- 
learning environments for fostering healthy social-emotional-behavioral 
development in these students. 

❖ Increased and sustained social-emotional-behavioral development by participating 
children. 

❖ Increased technical assistance, information dissemination and professional 
development opportunities for those who provide education and mental health 
services to these children, their teachers, and their families. 

❖ Increased collaborative planning with state, regional, and local service providers to 
implement model programs for children and youth who have severe social- 
emotional-behavioral disabilities. 

Together, these evaluation activities support the conclusion that the overall project mission 
to improve services for young children with SE/BD was achieved. 
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PROBLEMS ENCOUNTERED AND HOW THEY WERE SOLVED 



There were several unanticipated problems which influenced the direction of grant activities 
over this three-year period. These problems reflect issues and challenges in the early childhood field 
of severe social-emotional-behavioral disabilities and how they impact on outreach assistance for 
model implementation rather than problems specific to this project. Conditions that gave rise to these 
problems and how the project responded are described below, by these specific outreach activities: 
(1) Dissemination of information and introductory training about the model to early childhood 
personnel and families of participating children; (2) Planning and model implementation at selected 
replication sites; (3) Pilot Program for Training Trainers; (4) Evaluation of project impact; and, (5) 
Interagency Collaboration. 

Problems in Dissemination of Information and Introductory Training and Project Response 

The extremely large volume of requests for general information about the model was not anticipated. 
Numerous requests were received from individuals, program administrators, professionals, parents, 
or direct service providers nationally and internationally. The solutions were (a) an expansion and 
redesign of print materials into electronic media, (b) design of a web site with links to other 
resources, (c) sharing of materials and collaboration for translations in response to international 
requests for materials and guidance in model replication, and (d) extensive telecommunications to 
reduce actual staff travel for model dissemination purposes. To accomplish these new directions, it 
was necessary to redesign job descriptions for both key staff and for new personnel, primarily for 

use of advanced electronic communications and computer graphics design. “ 

This broad shift to electronic media resulted in considerable project staff time and resource 
allocation toward upgrading dissemination materials and developing introductory products. The 
original proposal anticipated production of a series of video tapes highlighting developmental^ 
appropriate strategies to be used at each stage of development. Resources were redistributed to the 
production of two introductory videotapes: one for early childhood and one for elementary school. 
The remaining resources were used to develop our computer software and to design an on-line 
course module to teach the basics for model implementation. 
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Problems in Planning and Model Implementation and Project Response 

The project received many more requests for assistance in model implementation than could 
be provided. It would have been impossible to meet this need using the outreach model of 
intermittent, in-depth training and follow-up at participating site visits for observations and 
debriefings with every individual team at each requesting program. The solution was to carefully 
assess the solidity of fit between the model and the established program. Project staff planned with 
the site administrators for specific needs at each program. Training began with a limited number of 
teams who volunteered for a one or two year pilot effort and was expanded in Years 2 and/or 3 to 
include additional teams. Two of the 20 sites were not able to continue implementation activities 
after initial training, due to turnover of both top administrators and teaching teams. 

Another problem in planning involved local concerns about how much additional work and 
time would be required of participants. Already over-loaded with paper work and record keeping, 
this question was raised by staff at every site. A parallel concern was the question of "fit" between 
the model's assessment instrument for developing IEP/IFSPs, Functional Behavioral Assessments, 
Behavioral Intervention Plans and the site's own district requirements. These issues of balance 
between model requirements, limitations in project staff, overload of staff at local sites, and their 
expressed needs for inservice assistance were addressed during planning with administrators at each 
site. In the initial inservice training with participating teams, these issues were frequently revisited. 
Most, but not all, of the sites were able to blend model implementation requirements with local 
requirements. In each instance where it did not occur, local administrators and participating teams 
made the decision to include the Developmental Therapy - Teaching instruments for social- 
emotional-behavioral assessment as an add-on to local requirements for IEP/IFSPs, FBAs and BIPs. 

Turnover of staff is an on-going problem throughout the field. We encountered numerous 
instances of absenteeism, staff resignations during the school year, and extended illnesses, causing 
shifts in job assignments and changes in teams participating in project activities for in-depth model 
implementation. Site administrators expressed their concern over this dilemma, which had left them 
with new, inexperienced or untrained replacement staff throughout the year. This frequent turnover 
of staff gave rise to a need for repeated introductory training sessions on the basics of model 
implementation, while previously trained staff members were ready for advanced skill development. 

To address this training problem, two initiatives were taken: ( 1 ) a re-shaping of our technical 
assistance was undertaken during Project Year 1. Training by staff from NEC*TAS resulted in 
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expansion of our long-distance communication links and instructional options to the implementation 
sites. These included a web site, LIST-SERVE, video teleconferencing, and frequent phone 
consultation with site administrators and with participating teams and (2) a pilot leadership training 
program was designed to train local coordinators/supervisors at the implementation sites. The 
objectives were (a) to prepare these direct service leadership individuals to conduct introductory 
inservice training for their new staff and (b) to provide support to existing staff as they continued 
to acquire advanced skills for model implementation. 

Problems in the Pilot Program for Training Trainers and Project Response 

Leadership individuals responsible for coordinating and supervising model implementation activities 
day-to-day expressed the need for advanced skills and knowledge about the model. This was 
particularly evident to them between visits of the project instructor. Participating teams went to them 
for feedback, problem solving, support, and guidance as they worked to implement the model with 
children who had severe social-emotional-behavioral problems. With staff turnover, these leadership 
individuals also found themselves needing to repeat introductory level training for new personnel. 

To address this need, during the first year of the project we focused on identifying leadership 
personnel at participating sites who wanted extended training to become certified trainers. Standards 
for acceptance into the program and five rigorous performance standards for certification were 
established. (See description in Management Objective 7.) Individual training programs were 
designed to meet these standards, and training was implemented during the latter half of Project 
Year 1 . Leadership training included shared presentations with the project instructor at their local 
sites and at regional and state conferences. They also were required to observe and debrief with their 
own staff and the project instructor during site visits. Trainees expressed some concern that they 
would be unable to complete all of the requirements for certification during the project funding 
period but were assured that they could progress through their training at their own rate. At this time, 
it appears to take most individuals approximately two years to accomplish the certification 
requirements. 

Problems in Project Evaluation and Project Response 

Evaluation presented the greatest problems for the project. The evaluation design proposed in the 
original proposal was used; (See Appendix H) however, problems inherent in field based data 
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gathering presented obstacles requiring modifications in several of the proposed evaluation 
activities. Because the evaluation plan had both formative and summative aspects, evaluation was 
a significant, time-intensive, ongoing project activity. It became necessary to shift position 
responsibilities among project staff because the collection and maintenance of accurate field records 
became increasingly demanding and time consuming. 

The most difficult aspect ofthe evaluation design was the assurance of reliability and validity 
of the observational performance data collected on participating teams and the children they served. 
At every site, project instructors reported the same types of difficulty in observing and rating the 
teaching teams at work. For example, when an instructor arrived at a site to observe a team's 
performance, it was not unusual to find (a) the group on a field trip, (b) a high number of children 
absent, (c) a key member of the team absent, (d) a substitute for the lead teacher (e) a non- 
representative activity such as lunch or rest, (f) new staff, and/or (g) some children following a part- 
time schedule in an inclusive general education class, necessitating a split in the team as one staff 
person went along to assure that the inclusive experience was successful. 

Collection of reliable and valid data on the progress of children served during the project 
presented a different set of problems. One of the core requirements for model implementation is the 
accurate use of the DTORF-R rating procedures by the participating teams as they rate the social- 
emotional-behavioral development of every child in their group. The project staff did not do these 
ratings, but reviewed each team's completed ratings for accuracy. If discrepancies were evident, the 
instructor and the team met to review and revise the ratings. This procedure required the project 
instructor to have sufficient time when on site to observe each child in the program. The original 
evaluation plan specified collection of both baseline and intermittent DTORF-R measures on each 
child. Valid baselines were sometimes difficult to obtain from some teams because (a) they lacked 
a sufficient understanding of the instrument even though they had participated in the preliminary 
core content workshops, (b) project staff had difficulty obtaining the basic demographic information 
needed to describe the sample population, (c) carelessly completed ratings, (e) untrained staff 
participating in the rating, and/or (d) incomplete ratings. Collection of valid ratings repeated 
throughout the school year to document progress was also difficult as children (a) moved away, (b) 
were newly enrolled, (c) transferred to other programs, and/or (d) were absent during the rating 
periods. 

To assure reliability and validity of the data and confidence in the accuracy of the findings 
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for participating teams and children, we had to accept smaller numbers in our samples. While this 
approach may have introduced bias into the sample selection process, we chose to use samples 
which had reliable data, excluding those where data were incomplete or inaccurately collected. We 
believe the smaller samples are representative of the typical participants, children served, and sites. 

Another, less significant change in the original evaluation plan involved the discontinuation 
of administering the pre- post- knowledge test for participating teams. A decision was made at the 
end of the first year of the project that a participant knowledge test per se was not entirely suitable 
because of the intense focus of our technical assistance on actual, demonstrated skills and 
performance of participants. The paper and pencil test put some team members at a distinct 
disadvantage and resulted in low scores when their actual performance had demonstrated 
understanding of the content. We believe that effective performance on the DTRITS requires 
understanding of the Developmental Therapy-Developmental Teaching knowledge base and is a 
sufficient proxy for a knowledge test. 

Problems in Interagency Collaboration and Project Response 

Interagency collaboration activities received less proportional staff time than other efforts yet the 
rapidly expanding need for close communication among agencies serving this population could have 
justified a full-time staff position. Early childhood intervention programs are facing the challenge 
of very young disruptive children with extremely difficult social-emotional-behavioral problems 
manifested in inclusive early childhood settings, child care, and foster care. The increasing incidence 
and severity of such problems served to further expand the need for interagency collaboration at 
local, regional, and state levels, including university and other state agency training programs. The 
special education/general education initiative for inclusion also required considerable cross-agency 
work. In addition, mental health agencies were beginning significant expansions into the schools to 
provide mental health services to troubled children and their families. Payments for these services 
and coordination of treatment plans with educational plans became major interagency issues at 
several sites. 

Because the extent of this need was not anticipated in the original project proposal, it fell to 
core project staff to provide interagency collaboration in a limited way at state, regional and national 
levels, while our field-based instructors represented the project in coordination with local sites and 
agencies. They were asked frequently to attend meetings, participate in conferences, and make 
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presentations. While these activities somewhat reduced the time project instructors spent in direct 
on-site assistance to participating teams, they were able to contribute to solutions for service 
delivery and personnel issues. 

IMPLICATIONS FOR POLICY, PRACTICES, AND RESEARCH 

There is deep, widespread concern among early childhood personnel and parents about young 
children. Young children with and without disabilities are at risk for significant delays in developing 
social-emotional-behavioral competence. Daily life seems to be increasingly more difficult for many 
families trying to cope with the work force, poverty, and child rearing. When parents are stressed 
to their limits, the challenges of providing responsible boundaries, guidance, and care are affected. 
Children are vulnerable to this stress; it impacts how they behave, how they cope, and how they use 
their abilities in learning opportunities. Moreover, changes in the states' welfare programs have 
added another complexity to the task of providing safe, secure, nurturing environments in which 
young children can develop in wholesome ways. A greater number of young mothers are leaving 
home to join the workforce. This results in child care and early childhood programs full to 
overflowing with young children, sometimes under inadequately trained staff. 

Adding to these facts, reports of national statistics show increases in exposure of young 
children to acts of violence and in their participation in dramatic acts of violence. Reported 
violations included those by children against themselves and other children, children against 
teachers, and children against their own parents. This growing evidence has raised awareness of 
society’s failure to meet the social-emotional needs of its young people. The explosion of violent- 
acts in schools during the past three years caused a surge in requests for information about the 
Developmental Therapy-Developmental Teaching model. We believe that the volume of requests 
we received reflects the deep concerns of those who work and live with seriously troubled children 
on a daily basis. Many adults are simply unprepared for the levels of complexity presented by this 
group of young people. Not only parents, but even experienced professionals, are faced daily with 
hurdles requiring skills and understanding beyond the ordinary scope of parenting, mental health, 
and educational interventions. 
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Recommendations for the Field 

The complexity of troubled young people demands an equally sophisticated, multidimensional 
approach with shared values and standards that transcend races and cultures. Providing for 
complexities involved in effective special education for this group of children should be a central 
principle in policy and practice. Here are several recommendations that would follow from such a 
central principle: 

1 . Program missions should be grounded in well established complementary theories 
about how children develop mentally healthy personalities, and include learning, 
valuing, relating, behaving, basic thinking and problem-solving. 

2. Programs should be conducted with seamless components for mental health 
interventions, and include involvement with other major social institutions that shape 
children’s lives - families, childcare, law, government, recreation, and spiritual life. 

3. Assessments should be based on procedures shown to be reliable and valid for 
identifying a child's current assets in each of the areas addressed in the scope of the 
intervention program. 

4. In planning a child’s intervention program, defined procedures should be used for 
gathering and analyzing past experiences to more fully understand their impact on 
a child’s current status. 

5. Advanced skill training with demonstrated proficiencies in developmentally and 
emotionally appropriate practices, human relationships, and sustained practice of 
mission standards should be required for anyone working in the early childhood 
field. 

6. Criteria for a child's progress should be established with defined outcomes having 
practical and theoretical validity. 
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7. On-going inquiry into the presumed effectiveness of every practice with every child 
should be part of every program. 

The field should be held to such basic standards for all children with severe social-emotional- 
behavioral disabilities. 

Recommendations for Effective Outreach and Technical Assistance 

Project experiences, problems encountered, and feedback from front-line practitioners over the past 
three years suggest numerous ways to assist individuals and programs at the local level in meeting 
the needs of this difficult-to-serve group of young people. 

1 , Family involvement in intervention programs . We believe that by putting greater emphasis 
on parents as team members, outreach projects can contribute significantly to enhancing constructive 
family involvement for a child’s benefit. Family involvement was initially low at the participating 
sites, reflecting similar widespread problems in the field. We found we were able to make some 
change in attitudes and practices by placing this as a high priority. Specifically, we found that 
parents, encouraged by the local staff and our project instructors to participate in the basic skills 
workshops along with program staff, were responsive and could utilize the training at home. We also 
found that this co-participation built greater understanding between staff and families. Feedback 
from some project participants noted that parents who participated with their child’s team in training 
and in rating social-emotional-behavioral development were increasingly positive about their child ’ s 
abilities and potenti’aTfor progress in the program. Staff who work closely with parents are also more 
optimistic about children's prognosis for positive growth. 

2. Skilled local leadership. At sites where we had a leadership trainee actively participating 
in model implementation as a trainer-in-training, there was greater progress by the teams in 
demonstrating and sustaining effective model practices. There appeared to be more confidence 
among the direct service teams to attempt new or improved practices when the coordinator was 
actively involved both during the instructor’s site visit and during the interim between visits. 
Consequently, the assessment of children's progress was more accurately completed at sites where 
there was an active leadership person in training. We conclude that the greatest benefits from model 
implementation accrue when local programs have their own supervisors/coordinators trained to high 
levels of proficiency and knowledge about the model. 
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3. On-going site evaluations. On-going evaluation of each child’s progress is essential if 
program quality is to be maintained. Each site should work to accumulate a database to build their 
own normative expectations about child progress in that program. At sites where more frequent child 
progress data were collected, we observed that teams were able to modify their day-to-day practices 
with more precision as children moved forward. In contrast, at sites where assessments were made 
only at the beginning and end of the year, program practices were not as readily changed as children 
changed. With infrequent evaluation, children tend to plateau and a program could inadvertently 
contribute to a ceiling on greater social-emotional-behavioral development. However, teams in 
intervention programs are not typically enamored of data collection processes, justifiably, as 
additional paper work and accuracy are necessary. We found that considerable outreach effort needs 
to be put into helping local administrators and coordinators put basic evaluation procedures in place. 
When local staff and parents (a) see evaluation results in formats easy-to-understand and interpret 
at a glance, (b) receive supportive assistance in using the results in practical ways to improve 
classroom conditions, and (c) are assured that their own value is not threatened by the results, there 
appears to be greater commitment to being a part of ongoing program evaluation. 

Recommendations for Outreach 

The degree of flexibility in current OSEP guidelines for conducting discretionary grant-funded 
projects is reasonable and helpful to outreach activities. With a field changing as rapidly as it did 
during the three years of this project, flexibility in modifying staffing patterns, staff assignments, 
tasks to be accomplished, and procedures was essential for successful accomplishmenfof the 
overarching project goals. We found the meetings in Washington for Project Directors extremely 
helpful in keeping abreast of current trends and new innovations — especially those that focused 
on our area of severe social-emotional-behavioral disabilities and technical problems of 
documenting intervention effects. We also found that contact with the larger national technical 
assistance projects was most helpful, especially those that came from allied fields involved with 
mental health or technological issues. If grant officers were encouraged to visit funded projects and 
implementation sites, this could promote greater utilization of proven models and practices 
nationally. 

At present funding levels, only core outreach services with intermittent assistance over a 
three-year period can be provided. Yet, requests for assistance far exceed capacity to respond both 
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within specific sites where implementation is occurring and at new sites where entire school districts 
or mental health programs seek assistance. Expansion of federal funding for outreach, both in dollar 
amounts and in funding periods (preferably from three to five years) would enable outreach 
programs to expand, sustain on-going efforts, and increase the depth of skills at participating sites. 
Increased funding levels would also allow for increased FTE for greater project involvement in 
interagency planning at state, local, and regional levels. With educational reforms at high levels, it 
seems essential that model outreach programs contribute to planning for educational improvements. 

Finally, our single greatest number of requests for on-site outreach assistance came from 
direct service providers who wanted to see model practices in action. They expressed a need to 
observe model practices demonstrated effective with children who had challenging behaviors similar 
to the ones they experience daily. Whenever possible, project instructors identified staff in local 
programs that were demonstrating proficiency and success with model practices. Yet, other teams 
at the same program seldom had released time to observe these practices and learn from them. We 
were especially pleased to make arrangements for a few direct service providers to visit other 
programs to see model practices. However, these opportunities were infrequent because of local 
funding limitations. 

Project instructors were also asked frequently to demonstrate specific strategies when making 
site visits. We did not encourage this because of project focus on staff training and not direct service 
delivery. However, we endorse the idea that skill acquisition is easier when there are opportunities 
to observe and model effective practices. A multiplier effect is created when local demonstration 
sites are available for observing and modeling effective practices. We believe that observations of- 
successful practices by direct service peers is among the most effective and cost efficient activities 
for a model outreach program. However, current OSEP ceilings on funding levels for model 
outreach projects make it difficult, if not impossible, to mount a project component to provide these 
opportunities. We recommend several policy standards for OSEP to consider: 

/ Expand maximum amount available for project proposals to include a specific 
component for direct service demonstration activities. 

/ Encourage outreach projects to offer extended regional summer institutes where 
direct service providers could gain intensive experience as observers and team 
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members in a successful demonstration program. 



/ Allow outreach projects to offer stipends and expenses for participation at in-depth 
training institutes for implementing model practices. 

/ Allow funds for partial payments to demonstration teachers at model sites and related 
program expenses such as transportation of children to the demonstration site. 

/ Encourage project FTE for project instructors to coordinate direct service programs 
for children and supervise learning experiences for direct service trainees at summer 
demonstration components of the outreach project. 

/ Allow funds, including travel and substitutes, for direct service providers to visit 
demonstration sites for short visits. 

In summary, this project has shown that extended on-site, in-depth, extended outreach 
assistance will result in improved program quality and skill acquisition by direct service providers, 
supervisors, coordinators, children, and their families. The lesson to be learned is that even more can 
be gained from these expenditures in the future if closer links are made available between an 
outreach project, local implementation programs, and high quality demonstration programs. 
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Figures 1-5 
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Figure 1. Overview of Performance Indicators, Final Performance Report 

Oct. 1, 1997 - Sept. 30, 2000 



Management Objective 


States 

Reached 


Schools/Sites 

Served 


Individuals 

Reached 


Children 

Benefiting 

Directly 


Other Outcomes 


1. DISSEMINATION* 


38 states & 
Virgin 
Islands; 15 
other nations 


NA 


3,800 


NA 


1,875 awareness 
materials distributed; 
3,000 bookmarks; 
2,200 newsletters 
mailed; 200 
monographs 


2. PLANNING FOR 
MODEL 

IMPLEMENTATION* 


7 states 


20 programs 


346 


585 children 
with disabilities 


38 training agreements 


3. MODEL 

IMPLEMENTATION 
& REPLICATION 


7 states 


20 programs; 
hours of direct 
on-site 

consultation & 
instruction 


346 

individuals 

with 

extended in- 
depth training 


585 children 
with special 
needs 


18 programs continuing 
with model 
components after 
training 


4. INTERAGENCY 
COLLABORATION* 


7 states 


14 agencies 
and 

8 universities 


NA 


NA 


Additional funding 
support received from 
Georgia and 
Washington State 


5. PROFESSIONAL 
DEVELOPMENT* 


National 
outreach in 
1 5 states and 
District of 
Columbia 


1 16 workshops 
(See also 
Figure 5) 


1,646 

participants 
in workshops 


NA 


24 other professional 
activities 

(presentations/exhibits, 

publications) 


6. PRODUCT 
DEVELOPMENT* 


NA ' 


NA- 


NA- - 


NA 


Web site; 2 videos 
introducing the model; 
FBA software; internet 
course modules; 
monograph 


7. PILOTING 
TRAINING OF 
TRAINERS* 


6 states 


20 leadership 
trainees 


13 schools & 
agencies 


NA 


13 trainees generated 
extended outreach 


8. EVALUATION 

(See Tables 1-11) 


NA 


Model fidelity 
measured at 1 3 
replication sites 


Sample 
performance 
data analyzed 
for 97 direct 
service 
trainees 


Sample 

performance of 
277 children 
analyzed for 
social/emotional/ 
behavioral gains 


Data storage/retrieval 
systems established; 
satisfaction survey 
from participants; focus 
group feedback from 
leadership trainees 



^During Project Years 2 & 3 costs and resources were shared with our severe outreach project and training of trainers project 
respectively. 
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Figure 2. Early Childhood Sites Participating in Planning for Model Implementation by Year (N = 20) 
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Figure 2. Continued 
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Figure 3. Summary Characteristics of the 20 Sites 
Receiving In-depth Staff Training 



Site 


# 

Staff 

Trained 


Type of 
Service 


ft of 


AGES 


Age 

Info 

Missing 


Children 

Served 


3-5 

Pre-K 


6-8 
K - 3 rd 


Other 


1 


15 


Partial Inclusion 
Special Classes 


82 


26 




39 


17 


2 


14 


Special Classes 


62 


12 


43 




7 


3 


20 


Full Inclusion 
Partial Inclusion 
Special Classes 


88 


88 


' 






4 


30 


Special Classes 


51 




1 


43 


7 


5 


37 


Full Inclusion 
Special Classes 


49 


47 


1 


1 




6 & 7 
2 sites 


(a) 20 

(b) 10 
30 


Full Inclusion 


31 


31 








8 


17 


Full Inclusion 
Special Classes 


20 




11 


9 




9 


5 


Partial Inclusion 
Special Classes 


10 




7 


3 




10 


25 


Full Inclusion 
Partial Inclusion 
Special Classes 


8 




8 






11 


3 


Small Group Pull-out 
for Therapeutic 
Session (2 x wk) 


10 




7 


3 




12 


3 


Partial Inclusion 


6 




5 


1 




13 & 17 


(a) 

(b) 
16. 


Special Classes 


19 

6 


17 

5 


2 

1 






14 


12 


Full Inclusion 
Special Classes 


20 


20 








15 


16 


Full Inclusion 
Partial Inclusion 
Special Classes 


20 


20 








16 


78 


Full Inclusion 
Partial Inclusion 
Special Classes 


31 




12 


19 


d 


18 


9 


Partial Inclusion 
Special Classes 


32 


2 


28 


2 




19 


9 


Special Classes 


28 


1 


27 






20 


7 


Special Classes 


12 


9 




3 




TOTALS 


20 


346 




585 


278 


153 


123 


31 



♦Children above age 8 were severely developmental^ delayed an/or with Autism Spectrum Disorders and functioning at preschool developmental levels. 
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Figure 4a. Occupations of Respondents to Workshop Evaluations 
Years One and Two (N = 389) 



Occupation 


Number* 


% of 389 


Teacher 


127 


32.6% 


Paraprofessional 


97 


24.9% 


(e.g., aide, ed tech) 






Program Coordinator 


28' 


7.2% 


Psychologist 


18 


4.6% 


Parent 


16 


4.1% 


Program Director 


8 


2.1% 


Social Worker 


8 


2.1% 


Family Service Professional- 


8 


2.1% 


Resource Consultant 


7 


1.8% 


Case Manager 


5 


1.3% 


Speech-language Pathologist 


4 


1.0% 


Counselor 


3 


.8% 


Childcare Consultant 


3 


.8% 


Program Specialist 


2 


.5% 


Community Service Provider 


1 


.3% 


Associate Teacher 


1 


.3% 


Consultant 


1 


.3% 


Grant Project Coordinator 


1 


.3% 


Assistant Professor 


1 


.3% 


Counseling Intern 


1 


.3% 


Behavior Team 


1 


.3% 


Other not specified 


55 


14.1% 



* some participants indicated more than one occupation 
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Figure 4b. Occupations of Respondents to Workshop Evaluations 

Year Three (N = 1257) 



Occupation 


Number* 


% of 1257 


Teacher 


489 


39% 


Para professional 


178 


14% 


(e.g., aide, ed tech) 






Psychologist 


32 


3% 


Parent 


62 


5% 


Program Director/ Coordinator 


118 


9% 


Social Worker 


78 


6% 


Family Service Professional 


23 


2% 


Speech-language Pathologist 


17 


1% 


Counselor/Mental Health Specialist 


35 


3% 


Childcare Provider 


9 


.7% 


Community Service Provider 


7 


.5% 


Program Administrator £r Supervisor 


28 


2% 


Consultant 


5 


.4% 


Student 


10 


.8% 


Professor 


8 


.6% 


Health Care Professionals 


36 


3% 


Researcher 


5 


.4%" 


Other Not Specified 


164** 


13% 



*Some participants indicated more than one occupation 
**150 Evaluations forms did not include occupations 
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Figure 5. Professional Development Activities: Sample of 116 Presentations and Papers 
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Table 1. Observation Performance Ratings of 45 Direct Service Teams 



Participant 

Teams 


DTRJTS SCORES and 
Proficiency Levels Achieved 


Team 

ID 


n 

Individuals 


Highly Effective 
90 - 100% 


Effective 

70 - 89% 


Adequate 

50 - 69% 


Below Passing 
<50% 


0206 


3 






59 




0302 


■ 2 


100 








0301 


2 


93 








0308 


4 


97 








0304 


2 








45 


0309 


2 








33 


0503 


2 




76 






0501 


2 


100 








0507 


2 




75 






0504 


2 






65 




0506 


2 


95 








0713 


2 


97 








0711 


2 








33 


0715 


2 


93 








mi 


2 


94 








1109 


2 




79 






1110 


2 ' 


100 








1404 


2 


100 








1405 


2 


100 








1603 


2 






68 




1611 


4 




81 






1605 


2 


- 






46 


1602 


2 




86 






1604 


2 




80 






1612 


2 


90 








1613 


2 


92 








1608 


2 




88 






1606 


2 




73 




* 


1614 


2 




84 






1615 


2 


94 








1609 


2 




89 






1616 


2 




85 






1617 


2 








33 


1701 


2 






67 




1704 


2 




78 






1706 


2 


100 








1804 


3 








40 



Q 52 

ERIC 
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1805 


2 








45 


2001 


2 








49 


2302 


2 








34 


2401 


3 






58 




2402 


2 


- 




64 




2501 


2 




78 






2601 


2 






63 




2602 


2 








38 


2603 


2 








16 


TOTAL 
45 teams, 

97 individuals 


14 teams, 

30 individuals 


13 teams, 

28 individuals 


7 teams, 

16 individuals 


1 1 teams, 

23 individuals 
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Table 3. Pre/Post Knowledge Test Scores 



Name 


Pre-Test 


Post-Test 


Passed 


A 


69% 






B 


62% 


81% 


/ 


C 


77% 


82% 


/ 


D 


70% 


83% 


/ 


E 








F 


71% 


86% 


/ 


G 


70% 






H 








I 


74% 


91% 


V 


J 


62% 


81% 


V 


K 


61% 


79% 


V 


L 


54% 






M 


66% 


80% 


/ 


N 


55% 






0 


66% 


82% 


V 


P 


67% 






Q 


66% 


. 




R 


74% 






S 


85% 




/ 


T 


78% 
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Table 4. Baseline Characteristics of Children at 15 Representative Sites Where In-depth Training for Model Implementation Occurred. 

(N=534) 



total 


534 


74 


386 

148 


306 

140 

10 

38 

02 

37 


242 

138 

79 

08 

22 


20 


cs 


36 

16-51 


05 

07 


09 

03 


07 

02 

03 


Os 


1 28 


89 

45-128* 


23 

05 


04 

24 


13 

15 


00 


32 


91 

53-129* 


26 

06 


07 

22 

02 

01 


26 

01 

01 

04 


so 




110 

60-142* 


24 

07 


17 

11 

02 

01 


08 

23 


m 


20 


34 

12-62 


08 

12 


12 

02 

06 


18 

02 


Tf 


20 


41 

19-57 


14 

06 


15 

01 

04 


17 

02 

01 


10 


08 


90 

65-103 


07 

01 


08 


03 

01 

02 

01 


00 


20 


108 

73-149* 


15 

05 


17 

03 


05 

06 
04 
02 
03 


6 & 7 


ro 


48 

37-59 


22 

09 


26 

02 

03 


08 

02 

20 


IT) 


49 


49 

36-60 


40 

09 


23 

11 

01 

09 

05 


19 

03 

01 




to 


169 

69-253* 


43 

08 


21 

29 

01 


02 

26 

20 

03 


rD 


00 

00 


56 

32-75 


61 

27 


56 

14 
01 

15 

02 


65 

05 
04 

06 


rt 


[ 62 


36 

4-74 


29 

33 


46 

04 

01 

02 

09 


47 

01 

12 

02 




82 


77 

37-134* 


69 

13 


45 

19 

03 

11 

04 


07 

53 

11 

02 


Site 


c 


Age (mos.) 

age range (mos.) 


Gender: 

boys 

girls 


Race: 

Caucasian 

African-Black 

Asian-PI 

Hispanic-Latino 

Native American 

Multiracial 


Primary Disability: 
ED/BD 
Autism 

Developmental 

Intellectual 

Speech/Lang. 



O 

r- 



<sj 

•o 



E 

Q. 

O 




E 

3 



U 

CJ 

CL 

C/D 





0> 

t 

<u 

5/5 



u 

V) 



S/5 

03 

J= 



i CD 
■S CD 

L* 

U 



E 





total 

38 

01 

06 


214 

320 


54 

146 

236 

96 

02 


20 


12 

0 


05 

03 

04 


Os 


16 

12 


06 

05 

08 

09 


00 


09 

23 


04 

17 

09 

02 


16 


21 

10- 


01 

02 

14 

14 

01 


m 


06 

14 


05 

01 

14 


TT 


II 

60 


02 

12 

06 


10 

01 


04 

04 


01 

07 


00 


0 

20 


01 

09 

10 


6 & 7 
01 


24 

07 


07 

20 

04 


5 

26 


04 

45 


05 

19 

24 

01 




46 

05 


01 

29 

21 


3 

01 

01 

06 


29 

59 


12 

33 

37 

05 

01 


CM 


21 

41 


11 

24 

25 
01 




69 

£1 


01 

11 

50 

20 


Site 

Other 

None 

Not Known 


Secondary Disabilit 

Yes 

No 


Severity of Delay: 
None 
Mild 

Moderate 
Severe 
Not Known 
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Table 5. Children's Performance Scores and Paired Sample t-Tests at Baseline 
and at Time 2 During Staff Training 
for Model Replication at 13 of 20 Sites (N = 277) 



Site 


n 


Mean sd 

DTORF-R 
Baseline 


Mean sd 

DTORF-R 
Time 2 


Average 
Timelag 
(months) 
Baseline- 
Time 2 


sd 


t-value 


Sig 

(2 tail) 


i* 


64 


35.47 


21.66 


42.03 


22.42 


6.1 


2.53 


5.10 


.000 


2 


24 


37.29 


19.08 . 


45.67 


17.06 


5.5 


1.47 


5.17 


.000 


3 


47 


47.68 


18.48 


56.06 


18.75 


5.1 


2.20 


7.03 


.000 


4* 


23 


49.91 


28.05 


54.17 


26.36 


5.0 


2.73 


1.73 


.097 


5 


35 


41.40 


13.91 


49.26 


16.15 


4.5 


1.67 


4.33 


.000 


6 

7 


incomplete data 


















8* 


7 


73.28 


21.02 


80.57 


23.52 


6.7 


1.60 


1.61 


.158 


9 


Did not ] 


participate in evaluation activities 


10 


7 


65.14 


18.49 


72.14 


15.75 


3.9 


.69 


2.70 


.036 


11 


Did not ] 


participate in evaluation activities 


12 


incomplete data 


















13 

17 


incomplete data 


















14 


17 


43.00 


15.07 


51.58 


12.71 


5.4 


1.88 


7.86 


.000 


15* 


3 


25.33 


3.21 


42.66 


10.69 


9.3 


1.53 


2.30 


.148 


16 


16 


52.13 


33.19 


62.56 


35.55 


5.5 


.52 


3.57 


.003 


18 


10 


61.90 


25.39 


78.50 


23.99 


7.0 


1.46 


146 


.007 


19* 


11 


48.73 


44.22 


52.91 


48.56 


4.5 


1.13 


2,11 


.060 


20 


8 


46.25 


16.36 


57.13 


19.58 


3.5 


1.41 


3.59 


.009 


n = 277 



*Sites marked with asterisks serve children with Autism Spectrum Disorders. 
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Table 6. Paired t-Tests Comparing Extrapolated Scores and Actual Scores at Time 2 
at Sites Where Children Made Significant Gains (N = 228) 
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value 
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CO 
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Table 7. Level of Administrative Support for Replication 
at Sites Participating in Project Evaluation 



Site 


Score* 


1 


41 


2 


33 


3 


32 


4 


30 


5 


41 


6 


34 


7 


incomplete data 


8 


31 


9 


incomplete data 


10 


21 


11 


25 


12 


incomplete data 


13 


17 


17 


15 


14 


24 


15 


32 _ . 


16 


12 


18 


20 


19 


22 


20 


27 



* Score indicates number of administrative elements in place for model replication 
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Table 8. Early Childhood Outreach Project 
Workshops and Presentation Evaluation Summaries 
October 1, 1997 - September 30, 2000 (N = 1646) 
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Table 9. Evaluation of Training 
October 1, 1997 to September 30, 2000 



Participants’ rating of their Developmental Therapy - Developmental Teaching 
training experience (N=82) 



Please check the Developmental Therapy ■ Developmental Teaching (DT-DT) training 
activities you have participated in. Circle how helpful each was for you.* 



ACTIVITY 


N 


Average response 


Workshop 


74 


4.18 


Direct Assistance with your Classroom 


Observations of your class by a DT-DT trainer 


62 


4.22 


Direct feedback by a DT-DT trainer 


65 


4.39 


Written feedback from a DT-DT trainer 


53 


4.04 


Other (e.g., textbook, discussions, assessment instrument, 
Regional Associate training) 


10 


4.60 


Total 


82 


4.28 



* scale of 5 (very helpful) to 1 (not very helpful) 
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Appendix A 

Administrative Organization Unit 
Within the University 
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Appendix A. 

Organization of the 

COLLEGE OF FAMILY AND CONSUMER SCIENCES 
Academic Programs, Research, and Service 
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Appendix B. 

2000-2001 Schedule of Fees 




Appendix B. 

Developmental Therapy-Teaching Programs 




U.S. Office of Special 
Education Program* 



P O. Box 5 1 53 College of Family & Consumer Sciences Phone: 706-369-5689 

Athens, Georgia 30604-5153 University of Georgia Fax:706-369-5690 

e-mail: mmwood@arches.uga.edu 



2000-2001 Schedule of Fees 



Workshops 

• Initial Training 
•Topical workshops 


Consultant/Instructor’s fees* 
plus travel and per diem 


• Materials are provided by 
Developmental Therapy - 
Teaching Programs 


Inservice for Implementation: 
Year-long Assistance 






• Onsite needs assessment and training 


$150.00 per participating 


• Number and dates of visits 


plan. 


team (one-time fee) plus travel 


are negotiated by site and 


• Onsite assistance for model 


and per diem for 


instructor. 


implementation. 


instructors/consultants. 


•Training materials are 


•Tutorials: Observation and feedback 




in participating classrooms. 




provided by Developmental 


• Ongoing consultation regarding 




Therapy - Teaching 


model techniques and implementation 
strategies. 




Programs. 


•Analysis of student and teacher 




•Site is responsible for 


progress. 




providing curriculum 


•Year end evaluation and report with 




materials (=$ 100/team)** 


recommendations. 




and release time for 
participating staff. 



* All consultants and instructors have successfully completed the National Developmental Therapy Leadership Training 
Program and are certified as both Developmental Therapy - Developmental Teaching demonstration teachers and as staff 
instructors. The per consultant/instructor fee for initial training or topical workshops is $500/day. For other forms of 
technical assistance, consultation, or insemce, Developmental Therapy-Teaching Programs provides instructors’ fees. Please 
call 706-369-5689 to discuss your training needs. 



** The curriculum guide, Developmental Therapy - Developmental Teaching (1996) by Mary M. Wood is available from PRO-ED, 
8700 Shoal Creek Boulevard, Austin, TK (800-897-3202). . 

The assessment instrument, Developmental Teaching Rating Objectives Form - Revised (1999), is available directly from the 
Developmental Therapy Institute, P.O. Box 5153, Athens, GA 30604 (706-369-5689). 
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Appendix C. 

Replication Standards 



Appendix C. 
Replication Standards 



Developmental Therapy-Teaching Programs 



P.O. Box 5153 

Athens, Georgia 30604-5153 



Phone: 706-369-5689 
Fax: 706-369-5690 



e-mail: mmwood@arches.uga.edu 



MINIMUM STANDARDS FOR MODEL REPLICATION 



Several minimal standards have been designated as necessary for replication: 

1. Staff training in the Developmental Therapy-Teaching practices. 

2. Accurate use of the Developmental Teaching Objectives Rating Form (DTORF-R) to assess 
each child’s social-emotional-behavioral status and identify an individual’s program 
objectives for these areas of development . 

3. Selection of services for children based on IFSP, IEP, or ITP goals, and utilization of the 
specified practices according to each child’s developmental stage. 

4. Staff performance of 75% effective or better on the Developmental Therapy Rating 
Inventory of Teacher Skills (DT/RITS). 

5. Team involvement of staff, family and teachers in each DTORF-R rating; and active 
participation by family members in the Developmental Therapy-Teaching program, 
whenever possible. 

6. Provision for concomitant enrollment of each child in an inclusive or integrated educational 
placement, whenever possible. 

7. An evaluation plan which includes the use of the DTORF-R and the DT/RITS, on a pre-post 
basis along with other evaluation data required for annual reports. In addition, sites provide 
child progress data from at least one standardized instrument, whenever possible. 

8. Evidence of administrative support for continuation of these program services, and a score 
of 16 or greater on the Administrative Support Checklist. 
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Appendix D. 

Content Map 
Instructional Modules 
for 

Participant Training 
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